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Second-Order Optimality Conditions for Broken 
Extremals and Bang-Bang Controls 


Nikolai P. Osmolovskii and Helmut Maurer 


Abstract We survey the results on no-gap second order optimality conditions (both 
necessary and sufficient) in the Calculus of Variations and Optimal Control, that 
were obtained in the monographs [31] and [40], and discuss their further develop¬ 
ment. First, we formulate such conditions for broken extremals in the simplest prob¬ 
lem of the Calculus of Variations and then, we consider them for discontinuous con¬ 
trols in optimal control problems with endpoint and mixed state-control constraints, 
considered on a variable time interval. Further, we discuss such conditions for bang- 
bang controls in optimal control problems, where the control appears linearly in the 
Pontryagin-Hamilton function with control constraints given in the form of a con¬ 
vex polyhedron. Bang-bang controls induce an optimization problem with respect to 
the switching times of the control, the so-called Induced Optimization Problem. We 
show that second-order sufficient condition for the Induced Optimization Problem 
together with the so-called strict bang-bang property ensure second-order sufficient 
conditions for the bang-bang control problem. Finally, we discuss optimal control 
problems with mixed control-state constraints and control appearing linearly. Tak¬ 
ing the mixed constraint as a new control variable we convert such problems to 
bang-bang control problems. The numerical verification of second-order conditions 
is illustrated on three examples. 

Keywords: broken extremal, piecewise continuous control, bang-bang control, sec¬ 
ond order necessary and sufficient conditions, critical cone, induced optimization 
problem. 
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1 Introduction 

We survey some main results presented in the recent monograph of the authors [40] 
(SIAM, 2012) and also some results obtained in the earlier monograph of Milyutin 
and Osmolovskii [31] (AMS, 1998). We discuss further developments of these re¬ 
sults and give various applications. 

Our main goal is present and discuss the no-gap second-order necessary and 
sufficient conditions in control problems with bang-bang controls. In [31], it was 
shown how, by using quadratic conditions for the general problem of the Calculus 
of Variations with regular mixed equality constraint g(t,x,u ) = 0, one can obtain 
quadratic (necessary and sufficient) conditions in optimal control problems in which 
the control variable enters linearly and the control constraint is given in the form of 
a convex polyhedron. These features were proved in Milyutin, Osmolovskii [31], 
who first used the property that the set ex U of vertices of a polyhedron U can 
be described by a nondegenerate relation g(u ) = 0 on an open set P2 consisting 
of disjoint open neighborhoods of vertices. This allowed us to develop quadratic 
necessary conditions for bang-bang controls. Further, in [31] it was shown that a 
sufficient condition for a minimum on ex U guarantees (in the problem in which the 
control enters linearly) the minimum on its convexification U. In this way, quadratic 
sufficient conditions for bang-bang controls were obtained in Osmolovskii, Maurer 
[40]. This property, which is not discussed in the present paper, constitutes the main 
link between the second order optimality conditions for a broken extremals in the 
Calculus of Variations and the second order optimality conditions for bang-bang 
controls in optimal control. 

The paper is organized as follows. In Section 2, we formulate no-gap second 
order conditions for broken extremals in the simplest problem of the Calculus of 
Variations. In Section 3, we consider such conditions for discontinuous controls 
in optimal control problems on a fixed time interval with endpoint constraints of 
equality and inequality type and mixed state-control constraints of equality type. 
In Section 4, we present an extension of the results of Section 3 to problems on 
a variable time interval. In Section 5, we discuss no-gap conditions for bang-bang 
controls. Bang-bang controls induce an optimization problem with respect to the 
switching times of the control that we call the Induced Optimization Problem (IOP). 
We have shown in our monograph [40] that the classical second-order sufficient 
condition for the IOP, together with the so-called strict bang-bang property, ensure 
second-order sufficient conditions for the bang-bang control problem. We discuss 
such conditions in Section 6. 

In the next two sections, the theoretical results are illustrated by numerical ex¬ 
amples. Namely, in Section 7, we study the optimal control of the chemotherapy of 
HIV, when the control-quadratic objective in [21] of L 2 -type is replaced by a more 
realistic L 1 -objective. In Section 8, we consider time-optimal controls in two mod¬ 
els of two-link robots; cf. [40]. Finally, in Section 9, we discuss optimal control 
problems with running mixed control-state constraints and control appearing lin¬ 
early. Taking the mixed constraint as a new control variable we convert such prob- 
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lem to a bang-bang control problem. We use this transformation to study extremals 
in the optimal control problem for the Rayleigh equation. 


2 Second-Order Optimality Conditions for Broken Extremals in 
the Simplest Problem of the Calculus of Variations 

2.1 The Simplest Problem in the Calculus of Variations 

Let a closed interval [to,tf\, two points a.b g IR", an open set 3. C IR 2 " +1 , and a 
function L : J2 > IR of class C 2 be given. The simplest problem of the Calculus of 
Variations has the form 

(SP) Minimize ^(*(•)) := f L(t,x(t),x(t))dt, (1) 

Jt 0 

x(to)=a, x(tf)=b , (t,x(t),x(t)) £ J2. (2) 

We consider this problem in the space W “ of Lipschitz continuous functions. 
The last condition in (2) is assumed to hold almost everywhere. A weak mini¬ 
mum is defined as a local minimum in the space W l, °°. We say that a function 
x g W 1 ’°°([fo,t/],IR^- ) ) is admissible if x satisfies (2) and, moreover, there exists 
a compact set *€ C =2 such that (t,x(t),x(t)) € T? a.e. in [to,tf\- Set u := x and 
w = ( x,u ). We call u the control. 

Let an admissible function x°(t) be an extremal in the sense that it satisfies the 
Euler equation 

Jt Li=Lx ‘ (3) 

Here and in the sequel, partial derivatives are denoted by subscripts. Set 
u°(t) :=x°(t), w°(t ) = ( x°(t),u°(t )). 

Let 

w(') = (x(-),u(-)) e #2 :=W l2 xL 2 , 

where W 1 ' 2 is the space of absolutely continuous functions with square integrable 
derivative and Lr is the space of square integrable functions. In the space # 2 , let us 
define the subspace 

JC := {w € #2 |-f(0 = ii(t) a.e., x(to) =x(tf) = 0} 

and the quadratic form 

&(w) = ftg (L ww (t,w°(t))w(t),w(t))dt 

= ft g f ((L xr x(t),x(t)) +2(L xu u(t),x(t)) + (L mi u(t),u(t))) dt. 
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The following theorem is well-known. 

Theorem 1. (a) If the extremal x° is a weak minimum, then Q ( vv) > 0 on dff. 

(b) If L2(w) is positive definite on Jff, then the extremal a 0 is a (strict) weak mini¬ 
mum. 

As is known, the quadratic conditions in Theorem 1 can be tested via the Jacobi 
conditions or via bounded solutions to an associated Riccati equation. 

For a broken extremal, the quadratic form has to be stated in a different way 
that allows for the formulation of no-gap necessary and sufficient second-order con¬ 
ditions. We will formulate these conditions and discuss their extensions to differ¬ 
ent classes of optimal control problems, including bang-bang control problems and 
problems with mixed constraints and control appearing linearly. 


2.2 Second-Order Optimality Conditions for Broken Extremals 

Let again x°(t) be an extremal in the simplest problem (1),(2), and let u°(t) = x°(t) 
be the corresponding control. Assume now that the control u°(t) is piecewise con¬ 
tinuous with one discontinuity point f* £ {toff). Hence, x°(t) is a broken extremal 
with a corner at /*. We say that t. is an L-point of the function u°(t) if there exist 
£ > 0 and C > 0 such that |«°(f) — )| < C\t — f*| for all t £ (f* — £,/*) and 

|w°(Y) — | < C\t — f*| for all t £ (/*,/* + e). Henceforth, we assume that f* 

is an L-point of the function u°(t). The following question naturally arises: which 
quadratic form corresponds to a broken extremal? 

Let us change the definition of a weak local minimum as follows. Set 0 := {f*} 
and define a notion of a 0-weak minimum. Assuming additionally that the control 
M°(f) is left-continuous at t*, denote by cl m°(-) the closure of the graph of u°{t). 
Denote by V a neighborhood of the compact set cl u°(-). 

Definition 1. We say that x° is a point of a ©-weak minimum (or an extended weak 
minimum ) if there exits a neighborhood V of the compact set cl u°(-) such that 
^(x) > / (x°) for all admissible x(t) such that u(t) £ V a.e., where u(t) = x(t). 

Clearly, we have the following chain of implications among minima: 

strong minimum =>- 0-weak minimum =>• weak minimum. 

Let us formulate optimality conditions for a 0-weak minimum. To this end, we 
introduce the Pontryagin function (Hamiltonian) 

H(t,x,u,X) = ku + L{t,x,u), 

where A is a row vector of the dimension n. Defining A (t) := —L u (t,x 0 (t),u°(t)), 
we have in view of the Euler equation (3): 

H u (t,x°(t),u°{t)Mt))= 0, - k{t)=H x (t,x 0 (t),u°(t),k(t )). 
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Denote by [A] the jump of the function A(f) at the point f*, i.e., [A] = A + — A - , 
where A~ = A(f*—) and A + = A(f*+). Let [H] stand for the jump of the function 
H(t) := f/(f,x°(f),M°(f),A(f)) at the same point. The equalities 

[A] = 0, [H} = 0 

constitute the Weierstrass-Erdmann conditions. They are known as necessary con¬ 
ditions for a strong minimum. However, they are also necessary for the 0-weak 
minimum. We add one more necessary condition for the 0-weak minimum: 

D(H) := -L+x°~ + LJx 0+ - [L,\ > 0, 

wherex 0_ =x°(f*—),LJ = L v (f*,x°(f*—)), [Lt] =Lf — Lf, etc. Clearly, 

D(H):=A + x°--A“x 0+ + [Ao], 

where Ao (t) = —//(f) (recall, that = H,(t) a.e.). Moreover, it can be shown 

that D(H) is equal to the negative derivative of the function 

AH(t) := X{t)[u°}+L{t,x 0 (t),u°(h+)) -L(f,x°(f),w°(f*-)) 

at f*. The existence of the derivative has been proved and, hence, this derivative can 
also be calculated as ) or as ^{AH){t t +). 

Now, let us formulate second order optimality conditions for a 0-weak mini¬ 
mum. Denote by P@W 12 the Hilbert space of piecewise continuous functions x(t), 
absolutely continuous on each of the two intervals [foT*) and anc l suc h that 

their first derivative is square integrable. Any x G P&W 1 ' 2 can have a nonzero jump 
[x] = x(f* + 0) — x(/* — 0) at the point f*. Let t be a numerical parameter. Denote by 
Z 2 (0) the space of triples z = (t,x,u) such that t g IR, x(-) G P@W X 2L , w(-) GL 2 , i.e., 

Z 2 (0) =IRxP 0 W 1 ’ 2 xL 2 . 

In this space, define the quadratic form 

Q@(z)=D(H)t 2 -2[L x \x av t+ [ f (L wn .(t,w 0 {t))w(t),w(t)}dt, 

Jt 0 

where [L x ] is the jump of the function L x (t,w°(t)) at the point f*, and 

x av = ^ (x(E~) +x(f*+)^. 

Set 

= {z G Z 2 (0): x(t) = u(t) a.e., [x] + [x°]f = 0, x(t 0 ) =x(tf) = 0}. 

Theorem 2. (a) If x° is a 0-weak minimum, then £2@(z) > 0 on JHq. (b) If £2@(z) 
is positive definite on then x° is a (strict) 0-weak minimum. 
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The proof of this theorem is given in [31]. Let us note that in [31], instead of t, 
we used a numerical parameter £ such that t = —C. This remark also applies to the 
subsequent presentation. 


3 Second-Order Optimality Conditions for Discontinuous 
Controls in the General Problem of the Calculus of Variations 
on a Fixed Time Interval 

3.1 The General Problem in the Calculus of Variations on a Fixed 
Time Interval 

Now consider the following optimal control problem in Mayer form on a fixed time 
interval [to,tf\. It is required to find a pair of functions w(t) = (, x(t),u(t )), 1 £ [?o,/y], 
minimizing the functional 


min f(w) := J(x(t 0 ),x(t f )) 


(4) 


subject to the constraints 

F(x(t 0 ),x(tf)) < 0, K(x(t 0 ),x{t f )) = 0, (x(t 0 ),x(tf))) £ &>, 1 

x(t)=f(t,x(t),u(t)), h(t,x(t),u(t)) = 0, (t,x(t),u(t)) £ £, j 

where & and £2, are open sets, x, u, F, K, /, and h are vector-functions. 

We assume that J, F, and K are defined and twice continuously differentiable on 
and / and h are defined and twice continuously differentiable on £1. It is also 
assumed that the gradients with respect to the control hi U (t,x,u), i = 1,... ,d(h) are 
linearly independent at each point (f ,x, u) £ such that h{t,x , u) = 0 (the regularity 

assumption for the equality constraint h(t,x, u) = 0). Here hi are the components of 
the vector function h and d(h) is the dimension of this function. 

Problem (1), (2) is considered in the space 


where n = d{x), m = d(u). Define a norm in this space as a sum of the norms: 

IMI := IMIu + ll«ll°°= K f o)|+ f f \x(t)\dt + esssup [f0i( j|«(f) I ■ 

o 

A weak minimum is defined as a local minimum in the space W. We say that w = 
(x, u) is an admissible pair if it belongs to W, satisfies the constraints of the problem, 
and, moreover, there exists a compact set ‘rf C =2 such that (t,x(t),u(t)) £ ’if for a.a. 
t£[tod f \. 

It is well-known that an optimal control problem with a functional in Bolza form. 
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min J(yf) := J(x(t 0 ),x(t f )) + [ f f 0 (t,x(t),u(t))dt, (6) 

Jt 0 

can be converted to Mayer form by introducing the ODE y = f){t,x.u), y(to) = 0. 


3.2 First-Order Necessary Conditions 

Let w° = (a°,m 0 ) be an admissible pair. We introduce the Pontryagin function (or 
the Hamiltonian) 

H{t ,x,u,X) = A f{t,x,u) 

and the augmented Pontryagin function (or the augmented Hamiltonian) 

H a (t,x,u,X, v) = H(t,x,u,X) + vh(t,x,u), 

where A and v are row-vectors of the dimensions d{x) = n and d(h), respectively. 
For brevity we set 


x 0 =x(t 0 ), x f =x(tf), T] = (xo,x f ). 

Denote by IR"* the space of ^-dimensional row-vectors. Define the endpoint La¬ 
grange function 

l ( 7 ], «o, a , J3) = oo/ (t?) + a F ( p ) + j3 K ( p ), 

where cto G IR, cc G (IR‘ /(F ^)*, ft G (IR c/ ^)*. Introduce a tuple of Lagrange 
multipliers 

p = (oo,a,/3,A(-),v(-)) 

such that A(-): [to,tf] —i► IR"* is absolutely continuous and v(-) : [toA/] —► (IR rf ^)* 
is measurable and bounded. Denote by A (l the set of the tuples p satisfying the 
following conditions at the point vv°: 

Oo > 0, a > 0, aF(ri°) = 0, «o +lfi^ a, + I/3./I = 1, 

A = A (f 0 ) = —Ixo, X{t f ) = l x/ , H“ = 0, 

where p° = (x°(to),x°(tf)), the derivatives l Xg and l Xf are at (r)°,ao,a,j3) and the 
derivatives //", H'j are at (f,x°(f),M°(f),A(f),v(f)), f G [*o,f/]. By a; and j3 j we 
denote the components of the row vectors a and /3, respectively. 

Theorem 3. //vv° z.v a weak local minimum, then Ao is nonempty. Moreover, Aq is a 
finite dimensional compact set, and the projector (Oo,a,/3,A(-),v(-)) —> (do, OC, j3) 
is injective on Aq. 

The condition Ao f- 0 is called the local Pontryagin minimum principle, or the 
Euler-Lagrange equation. Let Mq be the set of all p = (do,d,/3,A(-), v(-)) G Aq 
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satisfying the minimum condition for a.a. t £ [to,tf\' 

H(t,x°(t),u,X(t)) > H(t,x°(t),u°(t),k(t)) Mu £ U(t,x°(t)), 


where 


U(t,x) := {u £ IR m | ( t,x,u ) £ =2, h(t,x,u) = 0}. 


The condition Mq ^ 0 is called the (integral) Pontryagin minimum principle, which 
is a necessary condition for the so-called Pontryagin minimum. 

Definition 2. (A.A. Milyutin). The pair w° affords a Pontryagin minimum if for any 
compact set MS C J2 there exists e > 0 such that J? iw) > (vv°) for all admissible 

pairs w(t) = (x(t),u(t)), satisfying the conditions 

max|x(f) — x°(t)| < e, [ \u(t) — u°(t)\ < e, (t,x(t),u(t)) £ "if a.e. 


[to tf ] 


3.3 Second-Order Necessary Conditions 


Set 


W 2 :=W^{[t^t f }^')xL\[t 0 ,t f )^), 


Let Jtf be the set of all w = (x,u) £ #2 satisfying the following conditions: 

J'(rj°)fj<0, F'(T)°)f} <0 VieI F (r]°), K'(rj°)fj = 0, 

H t )=fw(t,w°(t))w(t), for a.a. t £ [to,tf], 
h w (t,w°(t))w(t) =0, for a.a. t £ [foT/L 

where rj = (x(to),x(tf)), /f(i7°) := {i ■ Ffr} 0 ) = 0} is the set of active indices. 
Obviously, Jff is a convex cone in the Hilbert space ’//A■ We call it the critical cone. 

Let us introduce a quadratic form in # 2 . For u £ A {) and vi> = (x, u) £ # 2 , we set 



where = l nn (ri 0 ,aQ,a, p), H‘Z(t) = H^t ,x°(t),u°(t),k(t),v(t)), and 

fj = (x(t 0 ) ,x(t f )). 

Theorem 4. Ifw° is a weak minimum, then the set Aq is nonempty and 


maxf2(/i,w) > 0 for all we JC. 

geAo 


The necessary condition for a Pontryagin minimum differs from this condition only 
by replacing the set Aq by the set Mq. 
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Theorem 5. Ifw 0 is a Pontryagin minimum, then the set Mq is nonempty and 

max Q. (fl, w) > 0 for all w £ JP. 
neM 0 

We now assume that the control u° is a piecewise continuous function on [toff] 
with the set of discontinuity points 0 = { 4 ,..., 4 }, to < r i < ... < t s < tf. We 
also assume that each t k £ 0 is an L-point of the function w° (see the definition 
in Section 2.2). In this case, the regularity assumption for h implies that, for any 
jtl = (ao,a,j3,A(-), v(-)) £ Ao, v(t) has the same properties as u°(t): the function 
v(f) is piecewise continuous and each of its point of discontinuity is an L-point 
which belongs to 0. By virtue of the adjoint equation A = —H°, the same is true 
for the derivative A(f) of the adjoint variable A. Now, the second-order necessary 
conditions can be refined as follows. 

For /i £ Mq, set 


D k (H a » ) = A A+ x 0A_ - A A “x 0A+ - [H^] k , (7) 

where [H"^] k is the jump of the derivative///'( 4 x 0 (f),M°(f),A(f), v(f )) at the point 
4 , and A A_ := A ( 4 —), A A+ := A(4+), etc. Note that Hf = —Ao, where Ao (?) = 
— H[t ), and hence — [H?] k = [Ao] A . Sometimes we omit the superscript jx in the 
notation D k (H a ^ ). 

We can calculate D k (H a ) using another method. Namely, D k (H a ) can be calcu¬ 
lated as the derivative of the “jump of H a ” at the point 4 . Introduce the function 

(A k H“)(t) = (A k H)(t) + (A k (vh))(t) 

= k{t)(f{t,x°{t),u 0k+ )-f(t,x°{t),u 0k -)) 

+ (v k+ h(t ,X° (t) ,u 0k+ ) — V A_ /!(f,A'°(f),M 0A ~)) . 

It can be shown that the function (A k H a )(t) is continuously differentiable at the 
point 4 £ 0, and its derivative at this point coincides with — D k (H “). Therefore, we 
can obtain the value of D k (H a ) by calculating the left or right limit of the derivatives 
of the function ( A k H a )(t) at the point 4 : 

D k (H a ) = -^-(A k H a )(t k ±). 
at 

For any p £ Mq, it can be shown that D k (H‘ ,lx ) > 0, k = 1,... ,s. Set 

Z 2 (0) := IR S x P @ W 1 ' 2 ([taff],TR n ) x L 2 (fo.r^lR”), 

where P®W 1 , 2 ([fo,f/];IR”) is the Hilbert space of piecewise continuous functions 
x(t), absolutely continuous on each interval of the set [toff] \ 0 and such that their 
first derivative is square integrable. Define a quadratic form in Z 2 (0) as follows 

= Zl =l {D k (H^)tl + 2[l] k xif k ) 

+ + ${ H Z(t)w(t),w{t))dt, 
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where z= (6,x,u), 9 = ij = (x(f 0 ),x(f/)), x k av = \{x(t k ~) +x(t k +)), 

w = (x, u). Define the critical cone in the same space by the relations 

/(7?°)i7<0, Fl(rj 0 )fi <0 \/i eI F (ri 0 ), K l (ri°)fi = 0, 
x(t) =f w {t,w°(t))w(t), for a.a. t G [f 0 ,//], 

[x]* + [x°] k t k = 0, k = 1,..., s 
h w (t,w°(t))w(t) = 0 for a.a. t G [?o,//]• 

Theorem 6. Ifw° is a Pontryagin minimum, then the following Condition holds: 

the set Mq is nonempty and 

max Q@(p,z) > 0 for all z & . 

geM 0 

Let us give another possible representation for the terms ( D k (H‘'^)i 2 + 2[X] k x^ v t k ) 
of the quadratic form Q(->(jl.z) on the critical cone .'/C. 

Lemma 1. Let jX G Mq and z = (0 ,x,u) G Then, for any k = the fol¬ 

lowing formula holds 

D k {H» )t\ + 2 \X] k 4 v t k = [Ao + Xx°} k t 2 + 2 [Ajc]*f*. (8) 

Proof. Everywhere in this proof we will omit the subscript and superscript k. Taking 
into account that 

D{H“) = X + x°- - X-x 0+ + [Ao], [x] + [x°]t = 0, 

we obtain 

D(H a )t 2 + 2[A]x ov F = f 2 [Ao] +f 2 ( A + x°~ — A~x 0+ ) + 2f[A]x av 
= t 2 [Ao] + 1 2 ([A* 0 ] - A+[x°] - A - [x°]) + 2f [A]f av 
= t 2 (jAo] + [Ax 0 ]) + A+[x]f+A~ [x]t+2t[X]x av 
= [Ao + Ax°]f 2 + (A+(x+ — x~) + A~(x + — x~) + (A + — A~)(jr +x + )^ t 
= [Ao + Ax°] t 2 +2 [Ax] t. 


3.4 Second-Order Sufficient Conditions 

Here, we will formulate sufficient optimality conditions, but only in the case of dis¬ 
continuous control m°. Let again w° be a piecewise continuous function with the set 
of discontinuity points 0 and let each t k G 0 be an L-point. A natural strengthening 
of the necessary condition sz/ in Theorem 6 turned out to be sufficient not only for 
the Pontryagin minimum, but also for the so-called bounded strong minimum. This 
type of minimum will be defined below. 
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Definition 3. The component x ; - of the state vector x is called unessential if the func¬ 
tions / and h do not depend on x, and the functions J, F, and K are affine in Xj(fo) 
and Xj(tf). Let x denote the vector composed by essential components of vector x. 

tf 

For instance, the integral functional ^ = / ffl ' fo(t,x,u)dt can be brought to the 
endpoint form: J! = y(tf) — y(fo), where y = fo(t,x,u). Clearly, y is unessential 
component. 

Definition 4. An admissible pair vv° affords a bounded strong minimum if for any 
compact set ^ C £2 there exists e > 0 such that ^ (w) > ^(w 0 ) for all ad¬ 
missible pairs w(t) = (x(f),w(f)) satisfying the conditions |x(fo) — x°(fo)| < £, 
max [, 0 (/ ] |x(f) — x°(t))| < £ and (t,x(t),u(t)) £ ft a.e. on [toff]. 

Definition 5. An admissible pair vv° affords a strong minimum if there exists £ > 0 
such that Jf(w) > ^(w 0 ) for all admissible pairs w{t) = (x(t),n(f)) satisfying the 
conditions |x(fo) — x°(fo)| < £ and max [ ?0 r ] |x(f) — x°(f))| < £. 

The following assertion follows from the definitions. 

Lemma 2. If there exists a compact set C =2 such that {(t,x,w) £ =2 : h(t,x,u) = 
0} C then the bounded strong minimum is equivalent to the strong minimum. 

Let us formulate sufficient conditions for a bounded strong minimum. For li G 
Mo, we introduce the following conditions of the strict minimum principle: 

(a ) H(t,x°(t),u,X(t)) > H(t,x°(t),u°(t),X(t)) 
for all t £ [to,tf\ \@,m / u°(t), u GU(t,x°(t)), 

(b) H(t k ,x°(t k ),u,h ( 4 )) > H k 

for all 4 £ 0 , u £ U(t k ,x°( 4 )), u ^ u°(t k —), u ^ u°(t k +), where 
H k := H(t k ,xP(t k ),u°{t k -),X(t k )) = H(t k ,x°(t k ),u°(t k +),X{t k )). 

We denote by Mq the set of all p £ Mo satisfying conditions (a) and (b). 

For p G Mo we also introduce the strengthened Legendre-Clebsch conditions: 

(i) for each 1 £ [toff] \ 0 the quadratic form 

(H a uu (t,x°(tU\t)Mt)Mt))u,u) 

is positive definite on the subspace of vectors u £ IR m such that 

h u (t,x°(t),u°(t))u = 0. 

(ii) for each 4 G 0, the quadratic form 

{H uu (4 > x° ( 4 ), m° ( 4 —), A ( 4 ), v (4 — )) m , w) 
is positive definite on the subspace of vectors u £ IR m such that 
h u (t k ,x° (t k ) ,u° (t k —))u = 0 . 
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(iiillhis condition is symmetric to condition (ii) by replacing (4—) everywhere by 
(4+)- 

Note that for each fi £ Mq the non-strengthened Legendre-Clebsch conditions hold, 
i.e., the same quadratic forms are nonnegative on the corresponding subspaces. 

We denote by Leg + (M () ) the set of all fi £ M {) satisfying the strengthened 
Legendre-Clebsch conditions (i), (ii), (iii) and also the conditions 

(ivP A '(//^) > 0forall/fc = 1,... ,s. 

Let us introduce the functional 

?(z) = (0,G) + (x(t 0 ),x(t 0 )) + [ f (u(t),u(t))dt, 

Jt 0 

where z = (9,x,u) and 0 = (ti,...,t s ). 

Theorem 7. For the pair w°, assume that the following Condition holds: the set 

Leg + (Mq ) is nonempty and there exist a nonempty compact set M C Leg + (Mq ) and 
a number C > 0 such that 


max £2&(p,z) > Cy(z) 

geM 

for all z £ dff. Then the pair w° affords a (strict) bounded strong minimum. 

The sufficient condition guarantees a certain growth condition for the cost 
which will be presented below. We define now the concept of the order function 
F(t,u). 

Assuming that the function u°(t) is left-continuous, denote by cl u°( •) the closure 
(in IR" 1+1 ) of its graph. Denote by cl m°(4-i, 4) the closure in IR'” +1 of the graph 
of the restriction of u°(t ) to the interval (t/t-i,tt), k= 1 , ... ,s + 1 , where f i+ i = tf. 
Then 

5+1 

cl «°(-) = U clM °(4-t.4)- 

k= 1 

Denote by if, k = 1......v + L a system of non-overlapping neighborhoods of the 

5+1 

compact sets cl m°(4-i , 4). Let Y = {_) if. 

k= 1 

Definition 6. The function r(t,u ) : IR 1+m —> IR is said to be an order function if 
there exist disjoint neighborhoods if of the compact sets cl u°(tk-\,tk) suc h that the 
following five conditions hold: 

(l)r(t,u) = \u — u°(t)\ 2 if (t,u) £ if, t £ (4_i,4), k= 1,... ,s +1; 

(2 )r(t,u) = 2\t — 4 | + |m — u ok ~ | 2 if (t,u) £ if,t > 4, k = 1,... ,s; 

(3 ) r(t,u) = 2\t — 4| + j u — u 0k+ \ 2 if (t,u) £ if + 1 , t < 4, k = 1,... ,s; 

(4) r(f,n)>0if (t,u)(£r; 

(5 ) r(t,u) is Lipschitz continuous on each compact set in IR 1+m . 
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For 5w(f) = ( 8x{t ), 8u{t)) in W we set 

y(8w) = ||5x||^+ [ r(t,u°(t) + 8u(t))dt. 

Jt 0 


We call y the higher order. This higher order corresponds to a typical minimum in 
the case of discontinuous control, and the order function F(f,v) corresponds to a 
typical Hamiltonian in this case. 

Note that the order j'J F (t,u°(t) + 8u(t))dt is much finer (smaller) than the func¬ 
tional J'J \8u(t))\ 2 dt. On the other hand, it can be proved that, on each compact set 
(in IR"'), the following lower bound holds for fj 2 F( t,u°(t ) + 8u{t))dt : 

(J' \ 8u b)\ dt r(t,u°(t) + 8u{t))dt , 

where C > 0 depends only on the compact set. 

Define the violation function at the point w°: 

d(F) 

V(8w) = (J(t 1 ° + 8i 1 )-J(r 1 0 )) + +Y,F i (r 1 0 + 8r,) + + \K(r 1 0 + Si 1 )\ + \\8x-8f\\ u 

i= 1 
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where 77° = ( x 0 (t 0 ),x 0 (t f )), Sri = (8x(t 0 ),8x(t f )), Sf = f(t,w° + 8w) - f(t,w°), 
Sw = (5x, Su), ||.| 1 is the norm in the space L 1 of integrable functions, and a+ := 
max{a,0} for a £ IR. 

Definition 7. We say that a bounded strong y- sufficiency holds at the point w° if 
there exists C > 0 such that for any compact set f C =2 there exists £ > 0 such 
that the inequality V{8w) > Cy(8w) holds for all 5w = (8x, Su) £ TS satisfying the 
conditions 


|5jr(f 0 )| < £, ||5 jc||<» < £, 

(; t,w°(t ) + 5w(f)) £ f, h(t,w°(t) + 8w(t)) = 0 a.e. 

Obviously, a bounded strong y-sufficiency implies a (strict) bounded strong mini¬ 
mum. Moreover, if the point vv° + 5w is admissible, then, obviously, V (5w) = 
(J(w° + Sw) — y(w°))+. Therefore, a bounded strong y-sufficiency implies the fol¬ 
lowing 

y-growth condition for the cost : there exists C > 0 such that for any compact set 
f C there exists £ > 0 such that 

/(w° + 8w) — J(w°) > Cy(Sw) 

for all Sw = (8x, Su) £ W satisfying (9) and such that (vv° + 5w) is an admissible 
pair. 

Theorem 8. The sufficient condition in Theorem 7 is equivalent to the bounded 
strong y-sufficiency. 

Theorems 6-8 were proved in [35]. Generalizations of these theorems for optimal 
control problem with regular mixed inequality state-control constraints were re¬ 
cently published in [37] and [38], An extension of the results of this section to 
problems on a variable time interval were obtained in [36]. 



4 The General Problem in the Calculus of Variations on a 
Variable Time Interval 

4.1 Statement of the Problem 

Here, quadratic optimality conditions, both necessary and sufficient, are presented 
in the following canonical problem on a variable time interval. Let ST denote a 
process (x(t),u(t) 1 1 £ [fo,f/]), where the state variable x(-) is aLipschitz continuous 
function, and the control variable «(•) is a bounded measurable function on a time 
interval A = [tofA. The interval A is not fixed. For each process ST, we denote here 
by 


77 = (t 0 ,x(t 0 ),tf,x(tf)) 
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the vector of the endpoints of time-state variable (t,x). It is required to find 27 
minimizing the functional 

min c /(^) :=J{tj) (10) 

subject to the constraints 

F(r,)<0, K(r,)=0, 77 e (11) 

x(t)=f(t,x(t),u(t)), h(t,x(t),u(t)) = 0, {t,x(t),u(t)) £ (12) 

where & and £2 are open sets, x, u, F, K, /, and li are vector-functions. 

We assume that the functions J, F, and K are defined and twice continuously 
differentiable on and the functions / and h are defined and twice continuously 
differentiable on i?. It is also assumed that the gradients with respect to the control 
hi U (t,x,u), i = 1,... ,d(h), are linearly independent at each point (t,x,u) £ £2- such 
that h(t,x,u) = 0. Here d(h) is a dimension of the vector h. 


4.2 First-Order Necessary Conditions 

We say that the function 77(f) is Lipschitz-continuous if it is piecewise continuous 
and satisfies the Lipschitz condition on each interval of the continuity. Let 

&=(x(t),u(t)\t€[to,tf]) (13) 

be a fixed admissible process such that the control «(•) is a piecewise Lipschitz- 
continuous function on the interval A with the set of discontinuity points 

0 = 4}, where t 0 <h < ... <t s <t f . 

In order to make the notations simpler we do not use here such symbols and indices 
as zero, hat or asterisk to distinguish the process from others. 

Let us formulate the first-order necessary condition for optimality of the pro¬ 
cess . We introduce the Pontryagin function FI (Hamiltonian), the augmented 
Pontryagin function Ft", and the endpoint Lagrange function l as in Section 3.2, 
but remember that now 77 = (to,xo,t/,Xf), Also we introduce a tuple of Lagrange 
multipliers 

p = (ao,a,/3,A(-),Ao(-),v(-)) (14) 

such that A(-) : A —> (IR^W)* and Ao(-): A —> IR 1 are piecewise smooth functions, 
continuously differentiable on each interval of the set A\0, and v(-): A —> (IR' 111 ' 1 )' 
is a piecewise continuous function and Lipschitz continuous on each interval of the 
set A\0. 

Denote by Mq the set of the normed tuples ll satisfying the conditions of the 
minimum principle for the process 3T\ 


16 


Nikolai P. Osmolovskii and Helmut Maurer 


«o>0, a> 0, aF(ri) = 0 , ao+Lai + 'L\Pj\ = 1, 

A = Ao = -Hf : H% = 0,teA\&, 

A(fo) = —/jet,> A(*/) = 4 /j Ao(fo) = Ao(f/) = l tf , (15) 

min H(t,x(t),u,k(t)) =H(t,x(t),u(t),k(t)), t GA\0, 
ueu(t,*(t)) 

H(t,x(t),u(t),k(t)) + ko(t) = 0, t £ A \0, 

where {/(f ,x) = {m £ IR^-"- 1 | h(t ,x, u) = 0, (; t,x , u) £ J2}. The derivatives Z AQ and l Xf 
are at (r),ao,a,j3), where T] = (to,x(to),tf,x(tf)), and the derivatives H x , H“, and 
H“ are at (t,x(t),u(t), A(f), v(f)), where t £ A \ 0. (Condition //" = 0 follows from 
other conditions in this definition, and therefore, could be excluded; yet we need to 
use it later.) 

Let us give the definition of Pontryagin minimum in problem (10)-( 12) on a vari¬ 
able interval [fo,f/]. 

Definition 8. The process ST affords a Pontryagin minimum if for each compact set 
7? C i? there exists e > 0 such that ^ (ST) > ^ (ST) for all admissible processes 
ST = (x(t),u(t) 1 1 £ [f 0 ,f/]) satisfying the conditions 

(a) |?o — to| < e, \tf — tf\<e, 

(b) max|x(f) — x(f)| < £, where A = [to, if], 
aha 

(c) f \u(t) — u(t)\dt < £, 
aha 

(d) (t ,x(t) ,u(t)) £ a.e. oni. 

The condition Mq / 0 is equivalent to Pontryagin’s minimum principle. It is the 
first-order necessary condition for Pontryagin minimum for the process ST. Thus, 
the following theorem holds. 

Theorem 9. If the process affords a Pontryagin minimum, then the set Mq is 
nonempty. 

Assume that the set Mq is nonempty. Using its definition and the full rank condition 
for the matrix li u on the surface h = 0 one can easily prove the following statement: 

Proposition 1. The set Mq is a finite-dimensional compact set, and the mapping 
p i—> (ao,a,/5) is injective on Mq. 

As in Section 3, for each p £ MqP^ £ 0, we define D k (H a V) by relation (7). Then, 
for each p £ Mq the following inequalities hold: D k (H a >0, k = 1, ... ,s. 


4.3 Second-Order Necessary Conditions 

Let us formulate a quadratic necessary condition for a Pontryagin minimum for the 
process ST as in (13). First, for this process, we introduce a Hilbert space STf. (&) and 
the critical cone SfS C J%(0). Again, we denote by P® IT 1,2 (4,1R rf W) the Hilbert 
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space of piecewise continuous functions x (-): A —■»JR d W, absolutely continuous on 
each interval of the set A \ 0 and such that their first derivative is square integrable. 
We set 

z= ( to,tf,9,x,u ), 


where 


to G IR 1 , t f G 1R 1 , 0 = (fi,... J s ) e IR V , x G P 0 W h2 (A,lR cl W), 




Thus, 

!£f 2 (0) := IR 2 x IR 1 x x L 2 (A,JR. d ^). 

Moreover, for given z we set 

w=(x,u),xo=x(t 0 ),Xf=x(tf), (16) 

X 0 =x(t 0 ) + t 0 x(t 0 ), Xf=x(t f ) +tfX(tf), i) = (to,xoJf,x f ). (17) 

By If(tj) = {( € {1,... ,d(F)j \ Fj(T]) = 0} we denote the set of active indices of 
the constraints Fj < 0. Let be the set of all z £ (0 ) satisfying the following 
conditions: 


J'(T])f) < 0, F'(ri)f) < 0 Vi G I F {r])- K’(r\)f] = 0, 

K*) = for a.a. f G [t 0 ,t f \, 

[x] 4 + [x] k tk = 0, k=l,...,s 

h w (t,w(t))w(t) = 0, for a.a. t G [fo,f/]. 

Clearly, is a convex cone in the Hilbert space Zi(0). We call it the critical 
cone. If the interval A is fixed, then we set q := (xo ,Xf) = (x(/o),x(?/)), and in the 
definition of JfT we have fo = f/ = 0, xo = xo, x/ = x/, and 77 = fj := (xo,x/). 

Let us introduce a quadratic form on iZ))©). For 71 G Mo and z G J£®, we set 

2fle(/x,z) = (l^^)+ f ,f mw(t),w(t))dt+ £ (D k (H u nt 2 k +2[X] k x k J k ) 

Jt o k= 1 v 7 

+ ^A(f 0 )x(fo) + Ao(fo))fo +2A(f 0 )x(fo)fo 

-(i(f/)x(f/) +Ao(f/))^-2A(f/)x(f/)7/, (19) 

where = 1^(7],do, a, J3), = H“ w {t,x(t),u(t),X(t),v(f)). We now for¬ 

mulate the main necessary quadratic condition of Pontryagin minimum in the prob¬ 
lem on a variable time interval. 

Theorem 10. If the process ST yields a Pontryagin minimum, then the following 
Condition holds: the set Mq is nonempty and 

max > 0 for all z G 

[ieM 0 
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Using (8), we can represent the quadratic form £2@ on JC@ as follows 

2Q @ (n,z) = j\H^v{t),w(t))dt+Y J ([Ax + Ao]^ + 2[Xx] k t^j 

+ (A(fo)x(fo) + Ao(fo)j?o + 2A(fo)x(fo)Fo 

- (l(tf)x(tf) + Ao(f/))fy - 2 X(tf)x(tf)t f . ( 20 ) 


4.4 Second-Order Sufficient Conditions 

Let us give the definition of a bounded strong minimum in problem (10)-(12) on 
a variable interval [fo,f/]. Let again x denotes a vector composed of all essential 
components of vector x (cf. Definition 3). 

Definition 9. The process ST affords a bounded strong minimum if for each compact 
set^ C 42- there exists e > 0 such that (St) > ^ (ST) for all admissible processes 
St = (x(t),u(t) 1 1 G [toff]) satisfying the conditions 

(a) |f 0 -fo| < e, \tf-t f \<£, \x(t 0 ) -x(f 0 )| < e, 

(b) max \x(t) — x(t)\ < £, where A = [to,If\, 

An A 

(c) (t,x(t),u(t)) G ^ a.e.onl 

The strict bounded strong minimum is defined in a similar way, with the nonstrict 
inequality ^ (St) > r Jt (ST) replaced by the strict one and the process St required 
to be different from ST. 

Let us formulate a sufficient optimality Condition which is a natural strength¬ 
ening of the necessary Condition The condition is sufficient not only for a 
Pontryagin minimum, but also for a strict bounded strong minimum. 

Theorem 11. For the process ST , assume that the following Condition holds: 
the set Leg + (Mq ) is nonempty and there exist a nonempty compact set M C 
Leg + (Mg ) and a number C > 0 such that 

max £2@{p,z) > Cy(z) (21) 

geM 

for all z G .yCp). Then the process ST affords a strict bounded strong minimum. 

Here the set Leg + (Af (| ) has the same definition as in Section 3.4. 
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5 Second-Order Optimality Conditions for Bang-Bang Controls 

5.1 Optimal Control Problems with Control Appearing Linearly 

Let again 3 denote a process (x(t),u(t) \ t £ [fo,f/]), where the time interval A = 
[t 0 ,t f ] is not fixed. As above, we set 

We will refer to the following control problem (22)-(24) as the basic control prob¬ 
lem'. 

Minimize Ji(3) := J(t]) (22) 

subject to the constraints 

F(ri)<0, K(rf) = 0, t]£3, (23) 

x(t) = f(t,x(t))+g(t,x(t))u(t), u(t) £ U, (t,x(t)) £ 3, to<t< tf. (24) 

Here x £ IR", u £ IR'”, F. K, and / are vector functions, g is n x m matrix function 
with column vector functions gi (t,x,u),... ,g m (t,x,u), 3 C IR 2 " +2 and 3 C IR" +1 
are open sets, U C IR"' is a convex polyhedron. The functions J, F, and K are as¬ 
sumed to be twice continuously differential on 3 1 , and the functions / and g are 
twice continuously differential on 3. 

A process 3 = (x(t),u(t) \ t £ [foTi]) is sa id to be admissible if x(-) is abso¬ 
lutely continuous, «(•) is measurable bounded and the pair of functions (x(t),u(t)) 
on the interval A = [fo,fi] with the end-points 77 = (fo,x(fo)? f i A(/i)) satisfies the 
constraints (23), (24). 

Let us give the definition of Pontryagin minimum for the basic problem. 

Definition 10. The process 3 = (x(t).u(t) \ t £ affords a Pontryagin mini¬ 

mum in the basic problem if there exists e > 0 such that J? ( 3 ) > J? ( 3 ) for all 
admissible processes 3 = (x(t),u(t) \ t £ [fo,f/]) satisfying 

ko-to|<£, |H-?i| <e, max |x(f) -x{t)\ < e, / \u(t)— u(t)\dt < e, 


where A = [t 0} tf ], A = 

Note that, for a fixed time interval A, a Pontryagin minimum corresponds to an 
L l -local minimum with respect to the control variable. 
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5.2 Necessary Optimality Conditions: The Minimum Principle of 
Pontryagin et al 

Let ff = ( x[t),u(t ) 1 1 £ [?o,f/]) be a fixed admissible process such that the control 
u(-) is a piecewise constant function on the interval A = [fo,f/]. Denote by 

0 = to<h < ... <t s <t f , 

the finite set of all discontinuity points (jump points) of the control u{t). Thenx(f) is 
a piecewise continuous function whose discontinuity points belong to 0, and hence 
x(t) is a piecewise smooth function on A. 

Let us formulate the Pontryagin minimum principle, which is the first-order nec¬ 
essary condition for optimality of the process The Pontryagin function has the 
form 

m 

H(t,x,u,X) = Xf(t,x) + A g(t,x)u = A f(t,x ) + ^ Xgi(t,x)ui, (25) 

i=l 

where A is a row-vector of the dimension d( A) = d(x) =n while x,u,f,F and K 
are column-vectors. The factor of the control u in the Pontryagin function is the 
switching vector function, a row vector of dimension d(u) = m. 

<y{t,x,X): = H u (t,x,u,X) =A g(t,x), 

<Ji(t,x,X): = H Ui (t,x,u,X) =Xgi(t,x), i=l,...,m, (26) 

aft ): = aft,x(t),u(t)). 

The endpoint Lagrange function is 

/(ao,a,j3,Tj) = a Q J{ri) + aF(ri) + PK(ri ), 

where a and j3 are row-vectors with d (a) = d(F), c/(/3) =d(K), and Oq is anumber. 
By 

JU = (oSo,0£,/3,A(-),Ao(-)) 

we denote a tuple of Lagrange multipliers such that A(-): A —> IR"* and Ao(-): A —> 
IR are continuous on A and continuously differentiable on each interval of the set 
A\0. 

Let Mq be the set of the normed collections u satisfying the conditions of Mini¬ 


mum Principle for the process 

d(F) d{K) 

ao>0, a >0, aF(rj) = 0, ao+ £ a,-+ ^ |j3 7 -| = 1, (27) 

i= i ;'= i 

A = -H x , Ao = -H t Vf G 4 \0, (28) 

X(to) = —l XQ , A (tf) = l Xf , Xo(t 0 ) —-l, 0 , Ao {tf)=l tf , (29) 

minH(t,x(t),u,X(t)) =H(t,x(t),u(t),X(t)) Vt€A\0, (30) 

ueu 

H(t,x(t),u(t),X(t)) + Ao(f) = 0 Vf £ A \ 0. (31) 
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The derivatives l X(] and l Xf are taken at the point (ao,a,j3,rj), and the derivatives 
H X) H, are evaluated at the point (t,x(t),u(t),X(t)). We use the simple abbreviation 
(t) for indicating all arguments (t,x(t),u(t),X(t)), t £ A\&. 

Theorem 12. If the process ST affords a Pontryagin minimum, then the set Mq is 
nonempty. The set Mq is a finite-dimensional compact set and the projector p t—> 
(ao,a,j3) is injective on Mo. 

In view of this theorem, we can identify each tuple p £ Mq with its projection 
(c*o, ck, j8). In what follows we set p = (ao,a,j3). For each p £ Mq and 4 £ 0, 
we define again the quantity D k (H fA ). Set 

(A k H)(t) = H(t,x(t),u k+ ,X(t)) — H(t,x(t),u k ~,X(t)) = c r(t) [u] k . (32) 


For each p £ Mq the following equalities hold 


—(A k H) I , =—(A k H) l 

dt K k '\t=h- k >\t 


dt 


=‘k+ , 


k = 1 ,... ,s. 


Consequently, for each p £ Mo the function ( A k H){t ) has a derivative at the point 
4 6 0. Set 

D k (H») = -j t {A k H){t k ). 

Then, for each p £ Mq, the minimum condition (30) implies the inequalities: 


D k {H»)> 0, k=l,...,s. 

As we know, the value D k (H^) can be written in the form 

D k {H^) = -H k+ H k ~ +H k ~H k+ - [H t } k = X k+ ^- - X k ~^+ + [Xo] k , 


(33) 


where H k ~ and H k+ are the left hand and the right hand values of the function 
H x (t) := H x (t,x(t),u(t),X(t)) at 4, respectively, [H t ] k is a jump of the function H t (t) 
at 4, etc. It also follows from the above representation that we have 

D k (H») = -6(t k ±)[u} k , (34) 


where the values on the right hand side agree for the derivative <7(4+) from the 
right and the derivative d(t k —) from the left. In the case of a scalar control u, the 
total derivative a t + O x x + <J~ A X does not contain the control variable explicitly and 
hence the derivative <j(f) is continuous at 4. 

Definition 11. For a given extremal process ST = { (x(t ).u(t)) \ t £ A } with a piece- 
wise constant control u(t) we say that u(t) is a strict bang-bang control if there exists 
p = (ao,a,P,X,Xo) £Mq such that 

Arg min u / e£/ (j(t)u' = [u(t-),u(t+)], t £ {t 0 ,tf] 


(35) 
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where \u(t—),u(t+)] denotes the line segment spanned by the vectors u(t—) and 
u(t+) in and a(t) := o(t,x(t),k(t)) = A (t)g(t,x(t)). 

Note that \u(t—),u(t+)] is a singleton {u(t)} at each continuity point of the control 
u(t) with u(t) being a vertex of the polyhedron U. Only at the points 4 G © does the 
line segment \u k , u k+ ] coincide with an edge of the polyhedron. 

It is instructive to evaluate the condition (35) in greater detail when the control 
set is the hypercube 

d(u) 

U = I [ 7 Mi,max], Mi,min ^ Mi, n(LX (i = 1 , .... cl (u j ). (36) 

i=l 

Let Si = {tk.i,k = 1 kj > 0, be the set of switching times of the i — th 

control component w,(f) and let a,-(f) = A,(t)gj(t,x(t)) be switching function for 
Uj, i = l,...,d(u). Then the set of all switching times is given by 


d(u) 

0 = {t h ...,t s } = U Sj, 

i=i 

and the condition (35) for a strict bang-bang control requires that 


(a) Oi(t) 7^0 MttfiSi (i=\,...,d(u)), 

(b) there is no simultaneous switching of control components Uj(t), 
i.e., Sj fl Sj = 0 V i 7 ^ j. 


Hence, the i-th control component is determined by the control law 


Ui(t) 


Mi ,mini if tf/(/) > 0 1 
Mi, maxj if Oi'(f) < 0 J 


Vf G 4 \Sj. 


(37) 


(38) 


Remark 1. There exist examples, where condition (a) in (37) is slightly violated 
as <Jj(tf) = 0 holds for certain control components u,: cf. the Rayleigh problem 
in Section 9.2 and the collision avoidance problem in Maurer et al. [29]. In this 
case, we require in addition that cr,(f/j ^ 0 holds to compensate for the condition 
<Ji(tf) = 0. This property is fulfilled for the Rayleigh problem in Section 9.2 and the 
control problem in [29]. 


5.3 Second-Order Necessary Optimality Conditions 

Here, we formulate quadratic necessary optimality conditions for a Pontryagin min¬ 
imum for a given bang-bang control. (Their strengthening yields quadratic suffi¬ 
cient conditions for a strong minimum.) These quadratic conditions are based on 
the properties of a quadratic form on the critical cone. 
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Let again T7 = ( x(t),u(t ) 1 1 £ [/o,fy-]) be a fixed admissible process such that 
the control u(-) is a piecewise constant function on the interval A = [to,tf\, and let 
0 = to < t\ < ... < t s < tf, be the set of discontinuity points of the 

control u(t). For the process .T, we introduce the space 3P(&) and the critical 
cone C as follows. Denote by F’@C 1 (4,IR a! ^) the space of piecewise 

continuous functions x(-) : A —> IR^*) that are continuously differentiable on each 
interval of the set A \ 0. For each x £ P@C l (A , IR^'F) and for 4 £ 0 we set = 
jc(4— ),x k+ = x(tk+) and [x] k = x k+ Now set 

z=(t 0 ,tf,0,x), 

where t 0 ,t f £ IR 1 , 0 = (t u ... ,t s ) £ JR 5 , x £ P @ C X (4,lR d W). Thus, 
z £ 3T(0) : = 1R 2 x IR S x P@C l {A ,R d W). 


For each z we set 

*0 =x(fo)+foi(f 0 ), x f =x{tf)+t f x(tf), ij = (t 0 ,xo,tf,x f ). (39) 

The vector fj is considered as a column vector. Note that 4 = 0, respectively, tf = 0 
for fixed initial time 4 , respectively, final time tf. Let 

I F (r,) = {i€{l,...,d(F)}\F i (r } )=0} 

be the set of indices of all active endpoint inequalities F{ < 0 at the point rj = 
(to,x(to),tf,x(tf)). Denote by the set of all z £ JT(0) satisfying the follow¬ 
ing conditions: 


J'(ri)fi<0, F/(rj)fj <0VieI F (rj), K'( 77)77 = 0, (40) 

x(t) = (fx(t,x(t)) +g x (t,x(t))u(t))x(t), (41) 

[jt] i +[jt]*/ t = 0, k = l,...,s. (42) 

It is obvious that is a convex, finite-dimensional and finite-faced cone in the 
space 3P(&). We call it the critical cone. Each element z £ Jf© is uniquely defined 
by the numbers 4 , if , the vector 0 and the initial value jf(fo) of the function x(t). Two 
important properties of the critical cone are formulated in the next two propositions. 

Proposition 2. For any ft £ Mo and z £ J£©, we have 

aoJ'(ri)ij =0, a i F i '(T])^= 0 Vt£/ F (rj). 

Proposition 3. Suppose that there exist p £ Mq with 04 > 0. Then adding the equal¬ 
ities cCjF'(r])fi = 0 Vi £ 7/r( 77 ) to the system (40)-(42) defining one can omit 
the inequality J' (r\)f) < 0 in that system without affecting JF@. 

Thus, is defined by conditions (41), (42) and by the condition 77 £ JPq, where 
is the cone in ]R 2f, (- v )+ 2 given by (40). But if there exists p £ Mq with Oq > 0, 
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then we can put 

X& = {\\ e iR rf M +2 | ^'( 77)77 <0, a,F/(r])f) = 0 Vt e//. (rj), K\r\)x\ = 0}. (43) 

If, in addition, a, > 0 holds for all i £ Ip(i 7 ), then JCff is a subspace in IR rf W +2 . 

Let us introduce a quadratic form on the critical cone JC® defined by the condi¬ 
tions (40)-(42). For each /i £ Mq and z £ df® we set 

2f2 0 (M,z) = (4^,n>+ [ tf (HH c x(t),x(t))dt+ £ (D'(^)Ff + 2[A]^ v 4) 

^0 £=1 V 2 

+ (a (t 0 )x(t 0 ) + Ao(fo)) ?0 + 2A(fo)x(fo)to 

- ^A(f/)x(f/) + Ao(f/)^f/ -2X(tf)x(tf)tf, (44) 

where cto, a,/3), //{* = H xx (t,x(t),u(t),A.(t)) and fj was defined in 

(39). Note that for a problem on a fixed time interval [to, f/] we have to = 1/ = 0. The 
following theorem gives the main second order necessary condition of optimality. 

Theorem 13. If the process ST affords a Pontryagin minimum, then the following 
Condition sf® holds: the set Mo is nonempty and ma Xfj,eM 0 f2 0 (/4,z) > 0 for all 
z £ dff®. 

Using ( 8 ), we can also represent the quadratic form 12® as follows 

2Q @ (p,z) = (tfrji j,fj)+ f * {Hf x x(t),x(t))dt+Y j ([?ix + Xo} k q + 2\Xx] k t k ) 

J tQ k=i ' ' 

+ (A(to)x(to) + Ao(fo)^fo T2A(to)x(to)to 

— (X(tf)x(t/) + Ao(tf)Jtj — 2X(tf)x(tf)tf. (45) 


5.4 Second-Order Sufficient Optimality Conditions (SSC) 

The state variable x; is called unessential if the function / does not depend on x,- 
and the functions h.J. K are affine inx,o := x;(fo) and x,/ :=xftj). Let x denote the 
vector of all essential components of state vector x. Let us define a strong minimum 
in the basic problem. 

Definition 12. The process :T affords a strong minimum if there exists e > 0 such 
that J* (&) > [ff) for all admissible processes & = (x(t),u(t) \ t £ [?o,f/]) sat¬ 
isfying the conditions 

(a) |?o —to| < e, \tf-tf\<e, |x(fo)-x(fo)| < e, 

( b ) max|x(f) — x(f)| < £, where A = [to,tf\, 

AnA 
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The strict strong minimum is defined in a similar way, with the non-strict inequal- 
ity Jf (57)> J- (SC ) replaced by the strict one and the process SC required to be 
different from SC. 

A natural strengthening of the necessary Condition sfi 0 of Theorem 13 turns out 
to be a sufficient optimality condition not only for a Pontryagin minimum, but also 
for a strong minimum. 

Theorem 14. Let the following Condition S8 0 be fulfilled for the process SC: 

(a) u(t ) is a strict bang-bang control (i.e., there exists p G Mq such that 
condition (35) holds), 

(b) there exists p G Mq such thatD k (H^) >0, k = 1 , . ..,s, 

(c) maxQ 0 (p,z)>O for all |G J£©\{0}. 

geM 0 

Then 57 is a strict strong minimum. 

Note that the condition (c) is automatically fulfilled if CXS 0 = {0}, which gives a 
first-order sufficient condition for a strong minimum in the problem. Also note that 
the condition (c) is automatically fulfilled if there exists p G Mq such that 

£2 0 (p,z) > 0 for all z G SrtS 0 \ {0}. (46) 

Clearly, there is no gap between the necessary condition sfi 0 of Theorem 13 and the 
sufficient condition °S 0 of Theorem 14. 


6 Induced Optimization Problem for Bang-Bang Controls and 
the Verification of SSC 

We continue our discussion of bang-bang controls. Second-order sufficient optimal¬ 
ity conditions for bang-bang controls had been derived in the literature in two differ¬ 
ent forms. The first form was discussed in the last section. The second one is due to 
Agrachev, Stefani, and Zezza [1], who first reduce the bang-bang control problem to 
a finite-dimensional optimization problem and then show that the well-known suffi¬ 
cient optimality conditions for this optimization problem supplemented by the strict 
bang-bang property furnish sufficient conditions for the bang-bang control problem. 
The bang-bang control problem, considered in this section, is more general than that 
in [1]. Following [39], we claim the equivalence of both forms of second order con¬ 
ditions for this problem. 
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6.1 Formulation of the Induced Optimization Problem and 
Necessary Optimality Conditions 

Let £? = ( x(t),u(t ) 1 1 G [fo,f/]) be an admissible process for the basic control prob¬ 
lem (22)-(24). We denote by ex U the set of vertices of the polyhedron U. Assume 
that u(t) is a bang-bang control in A = [tp,f/] taking values in the set ex U, 

u(t)=u k £zxU for t G (4-i,4)> fc = 1,... ,s+1, 

where p + i = Tf. Thus, 0 = {Pi,..., T s } is the set of switching points of the control 
«(•) with p < 4 + 1 for k = 0,1, Assume now that the set Mq of multipliers is 
nonempty for the process TY. Put 

x(t 0 )=x 0 , 9 = (Pi,...,4), ? = (Po,f/,x o ,0). (47) 

Then 0 G IR 4 , % G IR 2 x IR" x IR 4 , where n = d(x). 

Take a small neighborhood Y of the point C in IR 2 x IR" x IR 4 , and let 

C = 9) G Y, 

where 0 = satisfies to < t\ < ?2 < ... <t s <tf . Define the function 

u(t;to,tf, 9) by the condition 

u(t;to,tf, 9) = u k for f G (4_i,4), k = 1,... ,s+1, (48) 

where f J+ i = tf. The values u(tk\to,tf, 9), k = 1,... ,s, may be chosen in U arbitrar¬ 
ily. For definiteness, define them by the condition of continuity of the control from 
the left: k(4;*0i*/> 9) = w(4— 9),k=l,...,s. 

Let 9) be the solution of the initial value problem (IVP) 

x = f(t,x)+g(t,x)u(t\to,tf,6), t£[t 0 ,t f ], x(t 0 )=x 0 . (49) 

For each £ G Y this solution exists if the neighborhood "V of the point £ is suffi¬ 
ciently small. We obviously have 

x(t,To,Tf,xo,9)=x(t), teA, u(t-,?o,Tf,9)=u(t), t£A\0. 

Consider now the following finite dimensional optimization problem in the space 
IR 2 x IR" x IR 4 of the variables £ = (fo,f/,xo, 9): 

</(£) '■= J(to,xo,tf,x{tf’to,tf,xo,9)) —> min, 

■= F{to,xo,t f ,x(tf,to,tf,x 0 ,9)) < 0, 

&(0 ■=K(t 0 ,x 0 ,tf,x{tf,to,tf,xo,9))=0. 


(50) 
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u(t;to,tf,t\,t 2 ) 







to to t\ 

h h 

u 2 

t 2 tf tf 


Fig. 2 Bang-bang control with two switches. 


We call (50) the Induced Optimization Problem (IOP) or simply Induced Problem 
which represents an extension of the IOP introduced in [1], The following assertion 
is almost obvious. 

Theorem 15. Let the process fif be a Pontryagin local minimum for the basic con¬ 
trol problem (22)-(24). Then the point C, is a local minimum of the IOP (50), and 
hence it satisfies first and second-order necessary conditions for this problem. 


6.2 Second-Order Optimality Conditions for Bang-Bang Controls 
in Terms of the Induced Optimization Problem 

We shall clarify a relationship between the second order conditions for the Induced 
Optimization Problem (50) at the point ^ and those in the basic bang-bang con¬ 
trol problem (22)-(24) for the process -T. It turns out that there is an one-to-one 
correspondence between Lagrange multipliers in these problems and an one-to-one 
correspondence between elements of the critical cones. Moreover, for corresponding 
Lagrange multipliers, the quadratic forms in these problems take equal values on the 
corresponding elements of the critical cones. This allows to express the necessary 
and sufficient quadratic optimality conditions for a bang-bang control, formulated 
in Theorems 13 and 14, in terms of the IOP (50). Thus we are able to establish the 
equivalence between our quadratic sufficient conditions and those due to Agrachev, 
Stefani, Zezza [1], 











28 


Nikolai P. Osmolovskii and Helmut Maurer 


Let 57 = (x(t),u(t) j t £ [fo,f/]) be an admissible process in the basic problem 
with the properties assumed in Section 5.2 and let £ = (fo. tf. xq . 0 ) be the cor¬ 
responding admissible point in the IOP. The Lagrange function for the Induced 
Optimization Problem (50) is 

&(ji, 0 = 0) = «0^(C) + + P#(Q, (51) 

where p = (ao, a, j3), £ = (to,tf,XQ, 0), G = We denote by 57q the critical 

cone at the point £ in the IOP. Thus, 575q is the set of collections £ = (Fo,f/,xo, 0) 
such that 


</'(?)£ <o, &l(C)£<o,iei, Sf'(?)f = 0, (52) 

where 7 = {/1 ^, (C) = 0} is the set of indices of the inequality constraints active at 
the point £. For p £ Mo the quadratic form, of the induced optimization problem, is 
equal to {ff^(p, C)C, O- 

Let us formulate now second-order optimality conditions for the basic control 
problem in terms of the IOP. 

Theorem 16 (Second-Order Necessary Conditions). If the process 17 affords a 
Pontryagin minimum in the basic problem, then the following Condition s7q holds: 
the set Mo is nonempty and 

max (ffrr(PiC)CiC) >0 for all £ £ 5X5q. 

g£M 0 “ 

Theorem 17 (Second-Order Sufficient Conditions). Let the following Condition 
57 q be fulfilled for an admissible process 17 in the basic problem: 

(a) u(t) is a strict bang-bang control with finitely many switching times 4, k = 
l,...,s (hence, the set Mq is nonempty and condition (35) holds for some 
ft £ Mai, 

(b) there exists p £ Mq such that D k (H ^ ) >0, k = 1 ,... , s, 

(c) max(J?^(p,Q£,Q>0 for all ££JF6\{0}. 

g£M 0 

Then 57 is a strict strong minimum in the basic problem. 

Theorem 17 is a generalization of sufficient optimality conditions for bang-bang 
controls obtained in Agrachev et al. [1], Detailed proofs of Theorems 16 and 17 are 
given in [39] and in our book [40]. 

Remark 2. Noble, Schattler [32] and Schattler, Ledzewicz [43] develop sufficient 
optimality conditions using methods of geometric optimal control. There is some 
evidence that their sufficient conditions are closely related to the SSC in our work 
[39, 40]. However, a formal proof of the equivalence of both types of sufficient 
conditions has not yet been worked out. 
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6.3 Numerical Methods for Solving the Induced Optimization 
Problem 


The arc-parametrization method developed in [19], [24] provides an efficient method 
for solving the IOP. To better explain this method, for simplicity let us consider 
the basic control problem with fixed initial time to = 0 and fixed initial con¬ 
dition xo(0) = X(), and without inequality constraints < 0. For this prob¬ 

lem, we slightly change the notation and replace the resulting optimization vec¬ 
tor £ = (tf,ti,...,t s ) by the vector z = ( 4 ,..., 4 , 4 +i), 4+1 = tj. Instead of directly 
optimizing the switching times 4 , k = l,.,,,i, we determine the arc lengths (arc 
durations) 

<■?k ■ 6 ft 1 5 k (53) 

of bang-bang arcs. Hence, the optimization variable z = (ft,... ,4,4+1 )* is replaced 
by the optimization variable 

GlR i+1 , &:=**-**- 1 . (54) 


The variables z and c are related by a linear transformation involving the regular 
(s + 1) x (s + 1 )-matrix R : 


%=Rz, z = R 


/ 

1 

0 

0 ... 

0 

0 ^ 



1 

1 

0 ... 

0 

0 



0 

-1 

1 ... 

0 

0 

(55) 

V 

0 

0 

0 ... 

1 

1 ) 



In the arc-parametrization method, the time interval [ 4 ^ 1 , 4 ] is mapped to the fixed 
interval 


4 := 


k-1 k 
s-F 1 ’ s+ 1 


k — 1,... ,s + 1, 


(56) 


by the linear transformation 


t — Qk + bkT, T £ 4, 


(57) 


where 

a* =4-1-(*-!)&, b k = (s+l)% k . 


Identifying 


x(t) = x(ayt + b k z) = x(t) 

in the relevant intervals, we obtain the ODE system 


(58) 

(59) 


x(t) = (s+l)^ k (f(a k + b k t:,x(T))+g{a k + b k T,x(T:))u k ) for TGl k . (60) 

By concatenating the solutions in the intervals I k we get the continuous solution 
x(f) = x(f; q) in the normalized interval [0,1], When expressed via the new opti- 
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mization variable tq , the Induced Optimization Problem (IOP) in (50) is equivalent 

- 5+1 

to the following optimization problem (IOP) with tf = Y, '■ 

_ k=\ 

Minimize c f(%) :=J(t f ;x( 1,|)) 

subject to &(£):= K(tf;x( 1,£))=0. 

The Lagrangian function is given by 

■&(H,Z) = a0cf(Z) + l 3^(5)» M = (ao,/3). (62) 

Using the linear transformation (55), it can easily be seen that the SSCs for the 
Induced Optimization Problems (IOP) and (IOP) are equivalent; cf. similar argu¬ 
ments in [24]. To solve the (IOP), we use a suitable adaptation of the control pack¬ 
age NUDOCCCS in Btiskens [4],[5]. Then we can take advantage of the fact that 
NUDOCCCS also provides the Jacobian of the terminal constraints and the Hes¬ 
sian of the Lagrangian which are needed in the check of the second order condition 
in Theorem 17. 

In practice, we shall verify the positive definiteness condition (c) in Theorem 17 
in a stronger form. We assume that the multiplier can be chosen as u = (1, jS) and 
that the following regularity condition holds: 

ranked) = d(K). 

Let N be the n^ x (n^ — d(K)) matrix, n t e = n + s + 1 (where n = d(x)), with full 
column rank n { e — d(K), whose columns span the kernel of (q). Then condition 
(c) in Theorem 17, 

<J^(|,/3)fd)>0 Vf^O, # ? (|d = 0, (63) 

is equivalent to the condition that the projected Hessian is positive definite [6], 

N*J%z(£,P)N>0. (64) 


7 Numerical Example with Fixed Final Time: Optimal Control 
of the Chemotherapy of HIV 


The treatment of patients infected with the human immunodeficiency virus (HIV) 
is still of great concern today (Kirschner et al. [21]. The problem of determining 
optimal chemotherapies has been treated in Kirschner, Lenhart, Serbin [21] in the 
framework of optimal control theory. The optimal control model is based on a simple 
dynamic model in Perelson et al. [41] which simulates the interaction of the immune 
system with HIV. Kirschner et al. [21] use a control quadratic cost functional of L 2 - 
type. It has been argued in Schattler, Ledzewicz, Maurer [44] that in a biological 
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context it is more appropriate to consider cost functionals of L -type which are 
linear in the control variable. Therefore, in this section, we are studying an objective 
of L l -type, where the quadratic control is replaced by a linear control. The state and 
control variables have the following meaning: 

T(t) : concentration of uninfected CD4 + T cells, 

T*(t) : concentration of latently infected CD4 + T cells, 

T**(t) : concentration of actively infected CD4 + T cells, 

V It) : concentration of free infectious virus particles, 
u(t ) : control, rate of chemotherapy. 

The treatment starts at to = 0 and terminates at the fixed final time tf = 500 (days). 
Thus, the control process is considered in the interval [0,f/]. The dynamics of the 
populations are (omitting the time argument): 

dT/dt = - h t T + rT (l - r+ ^+ r " ) - kiVT > ^(0) = To, 

dT*/dt = k l VT-n T T*-k 2 T*, T*(0) = r 0 *, 

(65) 

dT**/dt = k 2 T*~ii b T**, T**( 0) = TJ* 

dV/dt = (1 - u)Nn b T** - k x VT - fi v V , V(0) = V 0 . 

The control constraint is given by 

0 < u(t) < 1 Vf G [0,tf], (66) 

where u(t) = 1 represents maximal chemotherapy, while u(t) =0 means that no 
chemotherapy is administered. Note that Kirschner et al. [21] consider the control 
variable v = 1 — u. It is convenient to write the ODE (65) as the control affine system 
(24), 

x = f(x)+g(x)u, x(0)=x 0 , x= (7’,7’*,r**,y) gIR 4 , (67) 

with obvious definitions of the vector functions f(x) and g(x). As in [21] we con¬ 
sider two sets of initial conditions which depend on the time at which the treatment 
starts after the infection. The following initial conditions are interpolated from [21] 
and have already been used in the [16]. 

Initial conditions after 800 days: 

T 0 = 982.8, T 0 *= 0.05155, T 0 ** = 0.0006175, Vo = 0.07306. (68) 

Initial conditions after 1000 days: 

To = 904.1, Tq = 0.3447, r 0 ** = 0.004167, V 0 = 0.4939. (69) 

The parameter and constants are taken from [21] and are listed in Table 1. 

Kirschner et al. [21] consider the following objective of Z. 2 -type which is quadratic 
in the control variable: 
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Parameters and constants 

Values 

Bt 

: death rate of uninfected CD4 + T cell population 

0.02 d _1 

Bt* 

: death rate of latently infected CD4+ T cell population 0.02 d 1 

Bb 

: death rate of actively infected CD4 + T cell population 0.24 d _1 

Bv 

: death rate of free vims 

2.4 d _1 

h 

: rate CD4 + T cells becomes infected by free virus 

2.4 x 10~ 5 mm 3 d~ ! 

k 2 

: rate T* cells convert to actively infected 

3 x 10- 3 mm 3 d 1 

r 

: rate of growth for the CD4 + T cell population 

0.03 d“' 

N 

: number of free virus produced by T** cells 

1200 

T 

1 max 

: maximum CD4+ T cell population level 

1.5 x 10 3 mm -3 

S 

: source term for uninfected CD4 + T cells, 
where s is the parameter in the source term 

10 d~‘ mm 3 

s/(l + V) 


Table 1 Parameters and constants 


rtf 

Minimize J(x, u) = / (— T(t)+Bu(t) 2 )dt (B = 50). (70) 

Jo 

Recall that the state variable is defined as x := (T,T*,T**,V) £ IR 4 . The optimal 
control that minimizes (70) subject to the constraints (65)-(69) is a continuous func¬ 
tion, since the associated Hamiltonian H(x, X , u) has a unique minimum with respect 
to u and the strict Legendre-Clebsch condition H uu = 2B > 0 holds. For the two sets 
of initial conditions (68) and (69), the numerical discretization and NLP approach 
using AMPL [13] and IPOPT [47] yields the optimal controls shown in Figure 7 
which were also obtained in Hannemann [16]. 


chemotherapy u 




0 100 200 300 400 500 

time t (days) 


Fig. 3 Optimal control for the L? functional (70). (left) begin of treatment after 800 days: initial 
conditions (68). (right) begin of treatment after 1000 days: initial conditions (69). 


Hannemann [16] showed that second-order sufficient conditions (SSC) are satis¬ 
fied for the controls displayed in Figure 3, since the associated matrix Riccati equa¬ 
tion has a bounded solution. Note that Riccati equations are discussed in [22, 28] 
and in our book [40], Chapter 4. 
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Instead of the L? functional (70) we consider now the functional of l) -type: 

r*f 

Minimize J\(x 1 u) = / (—T(t)+Bu(t))dt (B = 50). (71) 

Jo 

The Hamiltonian or Pontryagin function for this control problem is given by 

H(x 1 A,w) = — T + Bu + X(f(x) + g(x)u) , (72) 

where A = (A-/ , Ay-. Ay*-. Ay ) G IR 4 denotes the adjoint variable. The adjoint equa¬ 
tion and transversality condition are given by 

A = -H x (x,X,u), A(f/) = (0,0,0,0), 

since the terminal state x(tf) is free and the objective (71) does not contain a Mayer 
term. We do not write out the adjoint equation A = —H x (x, A, u) explicitly, since this 
equation is not needed in the sequel. The adjoint variables can be computed from the 
Lagrange multipliers of the associated Induced Optimization Problem (IOP). The 
switching function is given by 

a(x,X) = H„(x,X,u) = B — XyN^bT**, o(t) = cr(v(f),A(f)). (73) 


The minimization of the Hamiltonian with respect to u yields the switching condi¬ 
tion 


u(t) 


1, if ait) <0l 
0, if a{t) > 0 J 


(74) 


The control has a singular arc in an interval [fi, ^2] C [0,T], if c j(t) = 0 holds on 
[t 1 , tf\ ■ However, we do not discuss singular controls further because for the data 
in Table 1 we never found singular arcs. Indeed, the optimal control for the /J - 
functional (71) is the following bang-bang control with only one switch at t\ : 


u(t) 


1 for 0 < t < t\ 1 
0 for t\ <t<tfj 


(75) 


The terminal arc u(t) = 0 results from the terminal value c j(tf) = B > 0 of the 
switching function and the minimum condition (74). Hence, the IOP has only the 
scalar optimization variable 1 1 and thus the objective (71) reduces to a function 
/i(fi) = J\ (x. u). The arc-parametrization method [24, 40] and the code NUDOC- 
CCS [4] yield the following numerical results, where state variables are listed with 
8 digits and adjoint variables with 6 digits. 


Ji = -489810.49, h = 161.69571, 7%) = 983.49256, 

T*(t f ) = 0.04934668, = 0.00059104969, V{t f ) = 0.069932999, 

A r (0) = -0.125173, A r »(0) =-1.51988, A r «(0) = -2.94704, 

A v (0) = -0.449700. 


(76) 
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Fig. 4 Optimal solution for initial conditions (68): treatment starts after 800 days. Top row: (left) 
uninfected CDA + T cells, (right) latently infected CD4 + T * cells. Middle row: (left) actively in¬ 
fected CD4 + T** cells, (right) infectious virus particles V. Bottom row: (left) bang-bang control 
u, (right) bang-bang control u and (scaled) switching function cj in (73) satisfying the switching 
condition (74). 


The state and control variables and the switching function are displayed in Figure 
4. To verify that the second-order sufficient conditions (SSC) are satisfied for the 
computed extremal solution, we have to check the conditions of Theorem 17. The 
strict bang-bang property is satisfied, since we infer from Figure 4 (bottom, right) 
that the switching function satisfies 

a(f) <0 for 0<t<h, cr(fi)>0, cr(f) >0 for t\ <t <tj = 500. (77) 
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To verify the positive definiteness in condition (63), we note that the Hessian is sim¬ 
ply the second derivative of the objective J\ (t i) evaluated at the optimal switching 
time 1 1 = 161.695711 for which we find 


d 2 Ji 
dt 2 


1.5469 >0. 


Hence, the extremal solution (76 ) displayed in Figure 4 provides a strict strong 
minimum. 
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Fig. 5 Optimal solution for initial conditions (68): treatment starts after 800 days and and terminal 
condition T{tf) = 995. Top row: (left) uninfected CDA + T cells, (right) latently infected CD4 + T*. 
Middle row: (left) actively infected CD4+7’** cells, (right) infectious vims particles V. Bottom 
row: (left) bang-bang control u, (right) bang-bang control u and scaled switching function cr in 
(73) satisfying the switching condition (74). 
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Now we try to improve the optimal terminal value T (tf) = 983.493 of the uninfected 
CDCA+T cells. For that purpose we prescribe a higher terminal value and minimize 
the functional J\ (x. u) subject to the boundary condition 


T(t f )= 995. 


(78) 
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Fig. 6 Optimal solution for initial conditions (69): treatment starts after 1000 days and terminal 
condition T(tf) = 995. Top row: (left) uninfected CD4 + T cells, (right) latently infected CD4 + T*. 
Middle row: (left) actively infected CD4+7’** cells, (right) infectious virus particles V. Bottom 
row: (left) bang-bang control u, (right) bang-bang control u and scaled switching function a in 
(73) satisfying the switching condition (74). 
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The arc-parametrization method [24, 40] and the control package NUDOCCCS [4] 
furnish the results 

7! = -489044.529, h = 198.566451, T(t f ) = 995.0, 

T*(t f ) = 0.014576433, T**(t f ) = 0.00017436211, V(t f ) = 0.020625442, 

A r (0) = -33.7027, A T (t f ) = -211.377, A r *(0) = 28078.4 

A r **(0) = 2.76312, Ay(0) = 405.843. 

(79) 

Figure 5 displays the state and control variables and the switching function. Figure 
5 (bottom, right) shows that the strict bang-bang property (77) is satisfies. Condi¬ 
tion (c) in Theorem 17 holds because the critical cone = {0} contains of the 
zero element. Therefore, the extremal displayed in Figure 5 provides a strict strong 
minimum. 

Finally, we study the optimal solution for the initial values (69), when the treat¬ 
ment starts after 1000 days and, again, the boundary condition T(tf) = 995 is 
prescribed. The arc-parametrization method in [24, 40] and the control package 
NUDOCCCS yield the results 

Ji = -483480.87, t x = 254.5443, T{t f ) = 995.0, 

T*(t f ) = 0.014576939, T**{t f ) = 0.00017436817, V(t f ) = 0.020626158, 

A r (0) = -35.629, X T {t f ) = -214.021, A r *(0) = 4209.98, 

Aj»» (0) = 2.69187, A v (0) = 136.835. 

(80) 

Figure 6 depicts the state and control variables and the switching function. The 
SSC in Theorem 17 are satisfied, since the strict bang-bang property (77) holds and 
condition (63) holds in view of .Ao = {0}. Therefore, the extremal (80) provides a 
strict strong minimum. 


8 Numerical Example with Free Final Time: Time-Optimal 
Control of Two-Tank Robots 

In this section, we review the results in our book [40] on the optimal control of two- 
link robots which has been addressed in various articles; cf., e.g. [9, 14, 15, 33]. 
In these papers, optimal control policies are determined solely on the basis of first 
order necessary conditions, since sufficient conditions were not available. In this 
section we show that SSC hold for both types of robots considered in [9, 15, 33]. 

First, we study the robot model considered in Chernousko et al. [9]. Gollmann [15] 
has shown that the optimal control candidate presented in [9] is not optimal, since 
the sign conditions of the switching functions do not comply with the Minimum 
Principle. Figure 7 displays the two-link robot schematically. The state variables 
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Fig. 7 Schematical representation of a two-link robot. 


are the angles q\ and q 2 . The parameters I\ and h are the moments of inertia of 
the upper arm OQ and the lower arm QP with respect to the points O and Q, resp.. 
Further, let m 2 be the mass of the lower arm, L\ = \OQ\ the length of the upper arm, 
and L\ = QC the distance between the second link Q and the center of gravity C 
of the lower arm. With the abbreviations 

A = /[ + m 2 L] + I 2 + 2m 2 L\Lcosq 2 , B = I 2 +m 2 L\Lcosq 2 , 

Ri = u l +m 2 L\L(2q l +q 2 )q2smq2, R 2 = u 2 -m 2 L\Lq\ sin^ 2 , (81) 
D =I 2 , A = AD — B 2 , 

the dynamics of the two-link robot can be described by the ODE system 

qi = COi, <±>i = j(DR[ -BR 2 ), 

<72 = G>2, = j{AR2 —BRi) , 

where (0\ and fth are the angular velocities. The torques u\ and ui in the two links 
represent the two control variables. Therefore, the state variable and control variable 
are given by 

x= (qi,q 2 ,ah,a> 2 ) SIR 4 , u = (mi,m 2 ) e 1R 2 . 

The control problem consists in steering the robot from a given initial position to a 
terminal position in minimal final time If, 


tfi(O) =0, q 2 (0) =0, fi)i(0) = 0, ft) 2 (0) = 0, 

= —0.44, q 2 (tf) = 1-83, ( 0 \{tf) = 0, ft> 2 (f/) = 0. 


(83) 
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The control components are bounded by 

|hi(0I<2, MO I < 1 , te[0,tf]. (84) 

The Pontryagin function (Hamiltonian) is 

H = Ai®i + A 2 O 2 + — (DRi (mi) —BR. 2 (u 2 )) + ( AR. 2 (u 2 ) —BR\(u\)) . 

(85) 

The adjoint equations are rather complicated and are not given here explicitly. The 
switching functions are 


01 {x,X)=H Ul = ^D-^B, o 2 (x,X)=H U2 = ^A-^-B. ( 86 ) 

For the parameter values 

Li = 1, L = 0.5, m 2 = 10, /i =/ 2 = j, 

Gollmann [15] has found the following control structure with four bang-bang arcs. 


u(t) = (hi ( 0 , i/ 2 ( 0 ) = < 


(- 2 , 1 ) , 0 <t <t\ 
( 2 , 1 ) , h <t <t 2 

(2,-1 ) ,t 2 <t < ?3 
(- 2 ,- 1 ) ,/ 3 <t<tf 


,0<h<t 2 <h<tf. (87) 


This control structure differs substantially from the one in Chernousko et al. [9] 
which violates the switching conditions. Obviously, the bang-bang control (87) sat¬ 
isfies the assumption that only one control components switches at a time. Since 
the initial point (<71 ( 0 ), 172 ( 0 ),ft>i( 0 ),© 2 ( 0 )) is specified, the optimization variable 
in the IOP (61) is 


£ = (^ 1 > ^2)^3) ^4)) £l=*li %2=t 2 — h, E,3=h-t 2 , % 4 =tf~t 3 . 

Using the code NUDOCCCS we compute the following arc durations and switch¬ 
ing times 

h =0.7677893, & = 0.3358820, t 2 = 1.1036713, 

= 1.2626739, t 3 = 2.3663452, <^ 4 = 0.8307667, ( 88 ) 

tf = 3.1971119. 

Numerical values for the adjoint functions are also provided by the code NUDOC¬ 
CCS, e.g., the initial values 


Ai(0) = -1.56972, A 2 ( 0 ) = -0.917955, 
A 3 ( 0 ) = -2.90537, A 4 ( 0 ) = -1.45440. 


(89) 
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control u.| and switching function o 1 



control u 2 and switching function c> 2 



angle q., and velocity co-. 


angle q 2 and velocity co 2 



Fig. 8 Control of the two-link robot (81)—(84). Top row: (left) control u\ and scaled switching 
function <7i, (right) control m and scaled switching function o$. Bottom row: (left) angle q\ and 
velocity £0i, (right) angle qi and velocity Oh 


Figure 8 shows that the switching functions Of,, Oi comply with the minimum con¬ 
dition and that the strict bang-bang property (35) and the inequalities D k (H) > 0, 
k= 1,2,3 are satisfied: 


C7i(f)^0 for f^fi,f 3 , <j 2 ( 0^0 for 2 , 
cti( fi) < 0, cr 1 (f 3 )>0, & 2 ifi) > 0. 


For the terminal conditions (83) we obtain the Jacobian 


S*g(l) 


/-0.751043 0.0351060 0.258904 0\ 

3.76119 1.84929 -0.204170 0 

-0.326347 0.0770047 0.212722 -0.107819 
\ 1.26849 0.445447 -0.487447 -0.233634/ 


This square matrix has full rank in view of 

det^(|) =0.0766524^0, 
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which means that the positive definiteness condition (63) trivially holds. Thus we 
have verified first-order sufficient conditions showing that the extremal solution 
given by (87)-(89) provides a strict strong minimum. 

In the model treated above, some parameters like the mass of the upper arm 
and the mass of a load at the end of the lower arm appear implicitly in the system 
equations. The mass m\ of the upper arm is included in the moment of inertia I 2 and 
the mass M of a load in the point P can be added to the mass m 2 , where the point 
C and therefore the length L have to be adjusted. The length L 2 of the lower arm is 
incorporated in the parameter L. 

The second robot model that we are going to discuss is taken from Geering et 
al. [14] and Oberle [33]. Here, every physical parameter enters the system equation 
explicitly. The dynamic system is as follows: 

= G>1, ®i = i(Al22-BI n cosq 2 ), 

<?2 = ©2 — £<h, £02 = \ (Bin AI\ 1 cos q 2 ), 

where we have used the abbreviations 

A = 7i 2 £02 sin^2 + «i-«2, B = sin^ 2 + «2, 

4 = I\\I 22 d\2 cos ^ q 2 : In=h + (m 2 +M)L\, (91) 

hi = m 2 LL\ +ML\L 2 , h 2 = h+h +ML\ . 

h denotes the moment of inertia of the load with respect to the point P and ft >2 is 
now the angular velocity of the angle <71 +q 2 . For simplicity, we set h = 0. Again, 
the torques u\ and u 2 in the two links are used as control variables by which the 
robot is steered from a given initial position to a non-fixed end position in minimal 
final time tf. 


c/i( 0 ) = 0 , 

< 72 ( 0 ) = 0 , 
£01 ( 0 ) = 0 , 
£ 02 ( 0 ) = 0 , 


V (*1 ( h ) ~ x i (°)) 2 + (* 2 (£/) - * 2 ( 0)) 2 = r , 

qiitf) = 0 , 
£01 (t f ) = 0 , 
£02 ((f) = 0, 


where (211 (t),x 2 (t)) are the Cartesian coordinates of the point P, 


x\(t) = L\cosqfit) +L 2 cos(q\(t) + q 2 (t )), 
x 2 (t) = L l smq l (t)+L 2 sm(q 1 (t)+q 2 (t)). 


(92) 


(93) 


The initial point (xi (0),X2(0)) = (2,0) is fixed. Both control components are 
bounded, 

|«i(f)|<l, \u 2 {t)\<l, fG[0,f/]. (94) 

The Hamilton-Pontryagin function is given by 

H = Ai ©1 + A 2 (£02 — <0i) + x (A(u\,u 2 )I 22 — B(u 2 )h 2 cos q 2 ) 

{B(u 2 )I\\-A(ui,U 2 )h 2 cosq 2 ) ■ 


(95) 
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The switching functions are computed as 


= H UI = j (A 3/22 - A 4/12 cos ^ 2 ) , 

(96) 

o 2 (x,X)=H ll2 = j(A 3 (-/ 2 2 -/ 12 COS<7 2 ) + A 4 (/ll T/nCOS^)) ■ 

For the parameters values 

Li=L 2 = 1, L = 0.5, m\ = m 2 = M = 1 , I\ = / 2 = ^ h=0, r = 3, 

we will show that the optimal control has the following structure with five bang- 
bang arcs with 0 = to < t\ < t 2 < t 3 < f 4 < t 3 = tf : 


u(t) = (ui(t),u 2 (t)) = < 


(— 1 , 1 ) for 0 < t < t\ 
(- 1 ,- 1 ) for t\ < t < t 2 
( 1 ,- 1 ) for t 2 <t <t 3 > 
( 1 , 1 ) for t 3 <t <U 
(— 1 , 1 ) for f 4 < t < tf 


(97) 


Since the initial point (q\ (0 ),^2 (0), Ct>i (0), <x >2 (0)) is specified, the optimization vari¬ 
able in the optimization problem (50), resp., (61) is 


2 = (^1>&,^3,^4,^5), Zk=tk-tk- 1 , k= 1, - - - ,5. 

The code NUDOCCCS yields the arc durations and switching times 

r, =0.546174, = 1.21351, t 2 = 1.75968, 

£3 = 1 -03867, t 3 = 2.79835, | 4 = 0.906039, (98) 

U = 3.70439, £5 =0.185023, t f = 3.889409, 

as well as the initial values of the adjoint variables, 

1,(0) = 0.184172,1 2 (0) = -0.011125, 

A 3 ( 0 ) = 1.482636,1 4 (0) = 0.997367. { ' } 


The strict bang-bang property (35) and the inequalities D k (H) > 0, k = 1,2,3, hold 
in view of 


( 7 i(f )^0 for t^t 2 ,t 4 , CT,(f 2 )< 0 , cr,(f 4 ) > 0 , 
fr 2 ( 0^0 for t^t u t 3 , ct 2 (/i) > 0 , Ch{t 3 ) < 0 . 

For the terminal conditions in (92), the Jacobian in the optimization problem is 
computed as the (4 x 5)-matrix 
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control u 1 and switching function o 1 control u 2 and switching function o 2 





angle q-, and velocity co-, 


angle q 2 and velocity co 2 


Fig. 9 Control of the two-link robot (90)-(94). Top row. {left) control u i , {right) control 112 . Bottom 
row: {left) angle q\ and velocity ft)i, {right) angle qo and velocity ah. 


/ —10.8575 —12.7462 —5.88332 —1.14995 0\ 

0.199280 -2.71051 -1.45055 -1.91476 -4.83871 
-0.622556 3.31422 2.31545 2.94349 6.19355 

\ 9.36085 3.03934 0.484459 0.0405811 0 / 


which has full rank (64). The Hessian of the Lagrangian is given by 




/ 71.1424 90.7613 42.1301 8.49889 -0.0518216\ 

90.7613 112.544 51.3129 10.7691 0.149854 

42.1301 51.3129 23.9633 5.12403 0.138604 

8.49889 10.7691 5.12403 1.49988 0.170781 

\ -0.0518216 0.149854 0.138604 0.170781 0.297359 


This yields the projected Hessian (64) as the positive number 


A^.S?^(|,j3)jV = 0.326929. 


Hence, all conditions in Theorem 17 are satisfied and thus the extremal (97)-(99) 
yields a strict strong minimum. 

It is interesting to note that there exists a second local minimum with the same 
terminal time tf = 3.88941. Though the control has also five bang-bang arcs, the 
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control structure is substantially different from that in (97), 


u(t) = (ui(t),u 2 (t)) = 


( 1 ,- 

' 1 ) 

, 0 

<t< 

t\ 

- 1 ,- 

' 1 ) 

> h 

<t< 

h 

(-b 

» 1 ) 

! t 2 

<t< 

h 

( 1 : 

■ 1 ) 

) h 

< t < 

f 4 

( 1 .- 

1 ) 

) h 

<t< 

f / 


( 100 ) 


where 0 < t\ < t 2 < f 3 < f 4 < t^ = tf. NUDOCCCS determines the switching times 


h =0.1850163, t 2 = 1.091075, f 3 =2.129721, 

4 = 3.343237 , t/ = 3.889409, (101) 

for which the strict bang-bang property (35) holds and D k (H) > 0 for k = 1,2,3,4. 
Moreover, computations show that rank {§?£ (£)) = 4 and that the projected Hessian 
of the Lagrangian (64) is the positive number 

(|, fi)N = 0.326929. 

It is remarkable that this value is identical with the value of the projected Hessian 
for the first local minimum. Therefore, also for the second solution we have verified 
that all conditions in Theorem 17 hold, and thus the extremal (100), (101) is a strict 
strong minimum. The phenomenon of multiple local solutions all with the same 
minimal time tf has also been observed by Betts [3], Example 6.8 (Reorientation of 
a rigid body). 


9 Optimal control problems with mixed control-state constraints 
and control appearing linearly 

To the best of our knowledge, second order sufficient optimality conditions (SSC) 
for optimal control problems with mixed control-state constraints have only been 
studied for the class of regular controls, where the strict Legendre-Clebsch condi¬ 
tion holds. Such control problems have not yet been considered, when the control 
variable appears linearly in the system dynamics and in the mixed control-state con¬ 
straint. For a two-sided control-state constraint we will show that the constraining 
function itself can be taken as a new control variable, whereby the original control 
problem is transformed into a classical control problem with an affine control vari¬ 
able subject to simple control bounds. Hence, optimal controls for the transformed 
control problem are concatenations of bang-bang and singular arcs. The material in 
this section is based on our paper [27]. 
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9.1 Statement of the Problem and Transformed Control Problem 

For simplicity, we consider an optimal control problem with fixed initial time to = 0, 
fixed initial conditions and terminal equality constraints, and with a scalar control. 
Let x G IR" denote the state variable and it G IR be the control variable. The terminal 
time tf > 0 is either fixed or free. The dynamic equation and boundary conditions 
are 

x = f(t,x)+g(t,x)u, x(0)=xq, K(x(tf))= 0. (102) 

We consider a two-sided mixed control-state constraint which is affine in the control 
variable: 

a <a(x(t)) + b(x(t))u(t) < p for a.e. t £ [0,iy]. (103) 

The optimal control problem consists in finding a control u £ L°°([0,f/],IR) that 
minimizes the objective functional in Mayer form 

J(x,u)=J{x(tf)). (104) 

The functions f,g : IR" —>IR' ! , : IR” —*IR, / : IR" ->IRand/f: IR" -> IR d W (0 < 

d(K) < n) are assumed to be twice continuously differentiable. We remind the 
reader that a Bolza functional of the form 

9 

c f{x 1 u) =J(x(tf)) + J (fo(t,x)+go(t,x)u)dt. (105) 

o 

can be reduced to Mayer form by introducing the additional state variable y that 
solves the initial value problem y = fo(t,x) +go(t,x)u, y(0) =0 and minimizing thr 
functional J(x(t/)) +y(tf). 

The following regularity assumption will be assumed to hold for feasible trajec¬ 
tories: 

b(t,x(t))f 0 for t£[0,tf\. (106) 

This assumption allows us to introduce a new control variable v that is related to the 
control variable u as follows: 

v := a(x)+b(x)u, i.e., u — (v — a(x))/b(x). (107) 

The transformed optimal control problem consists in minimizing the objective (104) 
subject to the transformed dynamics 

x = f(t,x)+g(t,x)v, x(0)=x 0 , K(x(t f )) = 0, (108) 

where the transformed functions f,g are defined by 

f{t,x)=f(t,x)-g(t,x)a(x)/b(x), g(t,x)=g(t,x)/b(x). (109) 
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The mixed control-state constraint (103) then is equivalent to the simple control 
constraint 

a < v(f) < /3. (110) 

Thus, we can apply the second-order conditions developed in sections 5 and 6 to the 
transformed problem 


9.2 Numerical Example: Optimal Control of the Rayleigh 
Equation 

The Rayleigh equation describes oscillations of the electric current, resp., voltage 
in an electric circuit. The optimal control of the Rayleigh equation for a control- 
quadratic objective has been studied in Maurer, Augustin [23], Osmolovskii, Maurer 
[40] and Chen, Gerdts [ 8 ], where both simple control bounds and a mixed control- 
state constraint were investigated. 

Let xi denote the electric current and xn the voltage. The control u represents the 
voltage at the generator which steers the following dynamic equations: 

xi=x 2 , *t(0) = -5, 

X2 = — X\ +X2( 1.4 — 0 . 14 X 2 ) + U, X'2(0) = —5. 

We consider the mixed control-state constraint 

a<u+x 1 < /3 (a = -5, /3 = 0). (112) 

Various other bounds a and p have been studied in Maurer, Omolovskii [27]. The 
mixed constraint is a slight modification of the one considered in [23, 8 ]. The ob¬ 
jective is to minimize the quadratic functional 

J r (x,u) = J f (x\(t) 2 +x 2 (t) 2 )dt. (113) 

First, we consider this control problem with fixed terminal time tf = 4.5. Later, we 
shall prescribe the terminal condition 

xi(f/)=0 (114) 

and solve the control problem with free terminal time tf. 

According to (107), the new control variable v is given by 

V = K+Xl, u = v— Xl . (115) 

The Pontryagin function (Hamiltonian) with respect to the control v becomes 

H(x, A, v) = x| +X 2 + A 1 X 2 + A 2 (—xi +X2(1.4 — O.Mx?) + v — xi). (116) 


Second-Order Optimality Conditions for Broken Extremals and Bang-Bang Controls 


47 


The adjoint equations are 

A.i = -H Xl = -2xi + 2A 2 , 

Xi = -H X2 = -2x2 - Ai + A 2 (0.42xj — 1.4). 

For the first control problem with free endpoint x(tf) and fixed final time tf = 4.5, 
we get the transversality condition 


X l (tf)=0,X 2 (tf)=0, (118) 

while the second control problem with terminal constraint x\ (tf) = 0 and free ter¬ 
minal time tf gives the transversality conditions 

H(tf) = 0, X 2 (tf) = 0. (119) 

The switching function a = H v = A 2 determines the optimal control according to 

(P if X 2 (t) < 0, 

v(t) = < a if X 2 (t) > 0, (120) 

[ singular if X 2 (t) = 0 Vf Gl s C [0 ,tf\. 

We do not discuss singular controls further, because for the chosen bounds a = — 5 
and P = 0 in the mixed constraint (112) we obtain bang-bang controls. Singular 
controls for smaller values of a are discussed in [27]. 

In the first control problem with free terminal state x (tf) and fixed terminal time 
tf, we obtain a bang-bang control with three bang-bang arcs: 

f0 if 0 < t < t\ \ 

v(t) = i —5 if h <t <t 2 > ■ (121) 

[ 0 if t 2 < t < tf J 

The code NUDOCCCS [4] yields the following results: 

J(x,u) = 62.165171, 

h = 0.77996717, t 2 = 2.6835574, 
xi(t f ) = -0.40342897, x 2 (t f ) = -1.4332277, 

A! (6) = -13.364385, A 2 (0) = -5.591549. 

The corresponding extremal solution is shown in Figure 10. Next, we compute the 
Hessian of the Lagrangian for the IOP: 

j _ Z573.2366 458.8536\ 

L ' zl ~ ^458.8536 377.3994 ) ' 


Obviously, this matrix is positive-definite. Moreover, Figure 10 (top row, right) 
shows that the switching function a(t) = A 2 (t) satisfies the strict bang-bang prop¬ 
erty (37); cf. also Remark 5.1: 
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li(t) < 0 V 0 < f < ri, A2(fi)=0, (f i) > 0 , 

^2(0 > 0 Vfl<f<?2, ^-2(^2) = 0 , ^2(^2) < 0 ; 

A 2 (0 < 0 Vt2<t<tf, A 2 (f/)=0, ^(tf) > 0. 

Hence, the extremal shown in Figure 10 satisfies the SSC in Theorem 17 and thus 
is a strict strong minimum. 



state variables x 1 , x 2 


0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 

time t 


control v and switching function X 2 



time t 


adjoint variables ^ and X 2 


control u 




time t 


time t 


Fig. 10 Objective (113) and constraint — 5 < v = m + xi <0. Top row: (left) state variables x\,xi, 
(right) transformed control v and switching function a = A 2 . Bottom row: (left) adjoint variables 
kuh., (right) control u. 


Now we study the solution, when the terminal condition jq( tf ) = 0 is imposed and 
the terminal time tf is free. In this case we obtain a bang-bang control with only one 
switch: 


v(0 


0 if 0 < t < t\ 
—5 if t\ < t < tf 


(122) 


The code NUDOCCCS yields the following results: 


J{x,u) = 60.7269676, h = 0.834309981, t f = 2.3646877, 
X] (tf) = 0.0, X 2 (tf) = 1.60020237, 

At (0) = -13.4868419, Ai(f/) = -1.60020068, 

A 2 (0) = -5.72868282, A 2 (f/) = 0.0. 
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The corresponding extremal solution is shown in Figure 11. 


state variables x-, and x 2 


control v and switching function X 2 



time t 



time t 


adjoint variables k-,, X 2 


control u 


5 

0 

-5 


-10 


-15 




time t 


time t 


Fig. 11 Objective (113) and constraint — 5 < v = m + xi <0. Top row: (left) state variables xi,X 2 , 
(right) transfonned control v and switching function a = fa. Bottom row. (left) adjoint variables 
fa, fa, (right) control u = v — X\. 


To demonstrate that this extremal is a local optimum, we compute the Hessian of 
the Lagrangian for the IOP: 


/ 573.2366 458.8536 \ 
^458.8536 377.3994 ) ' 


Obviously, this matrix is positive-definite. Moreover, Figure 9.2 (top row, right) 
shows that the strict bang-bang property is fulfilled; cf. also Remark 5.1: 

fa(t)<0 V0<t<t u fa(ti)=0, A-2 (? i ) > 0 , 
fa(t)>0 V t\ < t <tf, fa(tf) = 0, < o. 

Hence, the extremal shown in Figure 11 satisfies the SSC in Theorem 17 and 
thus provides a strict strong minimum. 
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10 Conclusion 

We presented no-gap necessary and sufficient second-order optimality conditions 
for extremals with discontinuous controls in the simplest problem of the Calculus 
of Variations and the general optimal control problem with regular mixed constraint 
g(t,x, u) = 0 on a variable time interval [to,tf\. We formulated similar conditions for 
bang-bang controls in an optimal control problem with a Mayer functional, where 
the dynamical system is affine in control variable and the control constraint is given 
by a convex polyhedron. Bang-bang controls induce an optimization problem with 
respect to the switching times of the control, the so-called Induced Optimization 
Problem IOP. We showed that the classical second-order sufficient condition for 
the IOP together with the strict bang-bang property of the switching function en¬ 
sure second-order sufficient conditions (IOP) for the bang-bang control problem. 
The verification of SSC for bang-bang controls were illustrated on two numerical 
examples. First, we studied extremals in the optimal control of the chemotherapy of 
HIV. Then, following [40], we investigated extremals in time-optimal control prob¬ 
lems of two-link robots. We also discussed optimal control problems with running 
mixed control-state constraints and control appearing linearly. Taking the mixed 
constraint as a new control variable we converted such problems to bang-bang con¬ 
trol problems. As an example, we studied extremals in the optimal control problem 
for the Rayleigh equation. 

The results on SSC naturally lend themselves to sensitivity results for the IOP 
and the underlying bang-bang control problem using the well-known sensitivity re¬ 
sults for finite-dimensional optimization problems developed by Fiacco [12]. Sensi¬ 
tivity results for bang-bang controls may be found in Kim, Maurer [20] and Maurer, 
Vossen [30]. Sensitivity results also allow to develop real-time control techniques 
as indicated already in Biiskens et al. [7]. These issues will be the topic of a fu¬ 
ture paper. Related sensitivity results may be obtained for bang-singular controls as 
suggested in Vossen [45, 46]. This approach still needs a practical method of veri¬ 
fying the more abstract SSC for bang-singular controls given in Aronna, Bonnans, 
Dmitruk, Lotito [2]. 
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